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Purpose 

The Subcommittee on Investigations and Oversight on Sept. 10, 2009 convenes 
the first Congressional hearing to examine the role of risk modeling in the global 
financial meltdown. Risk models, and specifically a method of risk measurement 
known as Value-at-Risk, or VaR, are widely viewed as an important factor in the 
extreme risk-taking that financial institutions engaged in leading to last year’s eco- 
nomic upheaval. That risk-taking has led to hundreds of billions of dollars in losses 
to financial firms, and to a global recession with trillions of dollars in direct and 
indirect costs imposed on U.S. taxpayers and working families. 

Given the central role of credit in the economy, the ability of major financial insti- 
tutions to operate without assuming undue risks that gamble with the stability of 
the financial system, thereby endangering the broader economy, is of the utmost im- 
portance to both business and the public at large. The recent behavior by financial 
firms that are deemed “too big to fail” suggests that the financial system as cur- 
rently structured and regulated creates a “moral hazard” because firms can expect 
that they will be bailed out if their risk-taking fails to pay off. This is exactly what 
happened in the United States in October of 2008 with great consequences to the 
taxpayers, who have been called upon to shoulder much of the huge burden arising 
from financial firms’ underestimation of risk, poor judgment, and profligate behav- 
ior. Relied on to guide the decisions of both financial firms and federal regulators 
responsible for monitoring their soundness by ensuring that they have sufficient 
capital, the VaR, whether it was misused or not, was involved in inducing or allow- 
ing this situation to arise. 

Given this dual function, it is critical that the Subcommittee examine: the role 
of the VaR and related risk-measurement methods in the current world financial 
crisis; the strengths and weaknesses of, and the limits to, the usefulness of the VaR; 
the degree to which the VaR is understood, and may be manipulated, within the 
institutions where it is in use; and the capabilities and needs of federal supervisors 
who may be called upon to work with the VaR in carrying out their regulatory du- 
ties. From a policy perspective, the most important question is how regulators will 
use VaR numbers produced by firms and whether it is an appropriate guide to set- 
ting capital reserve requirements. 

This is the second in a series of hearings on how economic thinking and methods 
have been used by policy-makers both inside and outside of government. 

The VaR’s Origins and Use 

Risk assessment models in the financial industry are the product of advances in 
economic and statistical methods developed in the social sciences over the last fifty 
years. J.P. Morgan adopted these techniques in developing the VaR in the 1980s as 
a tool to measure the risk of loss to its traders’ portfolios. The VaR could produce 
a single number rating a trader’s (or, in aggregate, the firm’s cumulative) risk of 
loss of portfolio value over a specific period of time at a given level of confidence. 
The VaR provided managers a tool that appeared to allow them to keep a handle 
on the risks they were taking as financial instruments became more varied and 
complex and as assets became more difficult to value. Morgan decided to give the 
methodology of the VaR away, forming the now-independent RiskMetrics Group; 
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this resulted in the VaR rapidly becoming “so popular that it was considered the 
risk-model gold standard.” 1 

To put it very simply, the VaR captures the probability of outcomes distributed 
along a curve-most commonly a “bell” or normal distribution. It provides an answer 
to the question of, “what is likely to happen tomorrow to the value of an asset?” 
by drawing from historical performance data. The highest probability of tomorrow’s 
value is that it will be the same as today’s value; the next highest probability is 
for a very small movement in value up or down, and so on. The more radical the 
movement in value, the lower the probability of that occurring. A manager may ask 
for a projection of the potential loss of an asset or portfolio at the 95 percent or even 
the 99 percent confidence level. At those levels, a complete loss of value is unlikely. 
The complete collapse of an asset or portfolio’s value is not a l-in-100 event; such 
a collapse is more likely a l-in-500 or l-in-10,000 or event. The VaR is unlikely to 
warn, then, of great shifts in value. The danger to the financial firm or the commu- 
nity comes at the extreme margins of the distribution curves produced by the VaR. 
As a map to day-to-day behavior, the VaR is probably pretty accurate for normal 
times, but for asset bubbles or other “non-normal” market conditions, the VaR is 
likely to misrepresent risks and dangers. 

While the VaR was originally designed for financial institutions’ use in-house, it 
has subsequently been given a key role in determining capital requirements for 
large banks under a major multilateral agreement, the Basel II Accord, published 
in 2004. That same year, the U.S. Securities and Exchange Commission adopted a 
capital regime applying Basel II standards to the Nation’s largest investment 
banks, 2 a move that has been viewed as playing a role in those institutions’ subse- 
quent over-leveraging and liquidity problems. Those financial institutions assured 
regulators that the VaR was a way to see the level of risk they were taking on and 
a low VaR justified lower reserve requirements. (The terms of Basel II are currently 
being re-evaluated in light of the global economic crisis.) 

Along with extensive use, the VaR has come in for extensive criticism. Although 
its merits were debated at least as far back as 1997, 3 criticism of the VaR has 
mounted in the wake of last year’s collapse of such major financial institutions as 
Bear Stearns and Lehman Brothers. Among the allegations: that the VaR is inad- 
equate in capturing risks of extreme magnitude but low probability, to which an in- 
stitution may be left vulnerable; that this shortcoming may open it to manipulation 
by traders taking positions that seem profitable but whose risks they know the VaR 
is unlikely to pick up, and that such “gaming” can increase extreme risk; and that 
use of the VaR, derided for “quantifyfingl the immeasurable with great precision,” 4 
promotes an unfounded sense of security within financial institutions creating an 
environment where firms take on more risk than they would without the security- 
blanket of a VaR number. 

Those who advocate for the VaR argue that any misuse of the model is not the 
model’s fault and that it remains a useful management tool. VaR defenders’ argue 
that its purpose is “not to describe the worst possible outcomes;” 5 that it is essential 
to the ability of a financial institution to arrive at an estimate of its overall risk; 
and that in “computing their VAR[, institutions] are forced to confront their expo- 
sure to financial risks and to set up a proper risk management function,” so that 
“the process of getting to VAR may be as important as the number itself.” 6 Some 


ll ‘Risk Management,” by Joe Nocera, New York Times, Jan. 4, 2009. J.P. Morgan was not the 
only firm to look for statistical tools to measure the risks of their portfolios, however Morgan’s 
model became the most widely used. The model can be tweaked in many, many ways to meet 
the specific needs of a particular firm. 

2 “Alternative Net Capital Requirements for Broker-Dealers That are Part of Consolidated Su- 
pervised Entities; Supervised Investment Bank Holding Companies; Final Rules,” Securities and 
Exchange Commission, June 21, 2004, 69 FR 34428-72. (According to Aswath Damodaran, Pro- 
fessor of Finance at the NYU Stem School of Business, “The first regulatory measures that 
evoke Value-at-Risk, though, were initiated in 1980, when the SEC tied the capital require- 
ments of financial service firms to the losses that would be incurred, with 95 percent confidence 
over a thirty-day interval, in different security classes; historical returns were used to compute 
these potential losses. Although the measures were described as haircuts and not as Value or 
Capital at Risk, it was clear the SEC was requiring financial service firms to embark on the 
process of estimating one month 95 percent VaRs and hold enough capital to cover the potential 
losses.” Damodaran, “Value-at-Risk (VAR),” found at http:! / pages.stern.nyu.edu / ?adamodar / 
pdf lies / papers / VAR. pdf) 

3 “The Jorion-Taleb Debate,” DerivativesStrategy.com, April 1997, http:! / 
www.derivativesstrategy.com / magazine / archive / 1997 / 0497fea2. asp 

4 “Against VAR,” by Nassim Taleb, in "The Jorion-Taleb Debate,” ibid. 

5 “In Defense of VAR,” by Philippe Jorion, in “The Jorion-Taleb Debate,” ibid. 

6 Jorion, idem. 
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also argue that the VaR remains a useful tool for regulators to use as a baseline 
for establishing reserve requirements for “normal” times. 

Witnesses 

Panel I 

Dr. Nassim Nicholas Taleb, Distinguished Professor of Risk Engineering, Poly- 
technic Institute of New York University. 

Dr. Richard Bookstaber, Financial Author 
Panel II 

Dr. Gregg Berman, Head of Risk Business, RiskMetrics Group 
Mr. James G. Rickards, Senior Managing Director, Omnis Inc. 

Mr. Christopher Whalen, Managing Director, Institutional Risk Analytics 

Dr. David Colander, Christian A. Johnson Distinguished Professor of Economics, 
Middlebury College 
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Chairman Miller. Good morning, and welcome to today’s hear- 
ing: “The Risks of Financial Modeling: VaR and the Economic Melt- 
down.” 

Economics has not been known in the past for mathematical pre- 
cision. Harry Truman said he wanted a one-handed economist be- 
cause he was frustrated with economists who equivocated by saying 
on the one hand, on the other hand. George Bernard Shaw said 
that if all the world’s economists were laid end to end, they still 
wouldn’t reach a conclusion. And apparently no one is sure who 
first observed that economics was the only field in which it was 
possible for two people to share a Nobel Prize for reaching exactly 
the opposite conclusion about the same question. 

In the last 15 or 20 years, math and physics Ph.D.s from aca- 
demia and the laboratory have entered the financial sector. Quan- 
titative analysts, or ‘quants,’ directed their mathematical and sta- 
tistical skills to financial forecasts at a time when global financial 
markets were becoming more interdependent than ever before. 

The quants conceived such financial instruments as collaterized 
debt obligations, or CDOs, and credit default swaps, or CDSs, that 
would never have existed without them and their computers. They 
developed strategies for trading those instruments even in the ab- 
sence of any underlying security or any real market; for that mat- 
ter, in the absence of anything at all. They constructed risk models 
that convinced their less scientifically and technologically adept 
bosses that their instruments and strategies were infallibly safe. 
And their bosses spread the faith in the quants’ models to regu- 
lators, who agreed to apply them to establish capital reserve re- 
quirements that were supposed to guarantee the soundness of fi- 
nancial institutions against adverse events. It almost seemed like 
the economic models had brought the precision of the laws of phys- 
ics, the same kind of certainty about the movement of the planets, 
to financial risk management. Engineering schools even offered 
courses in “financial engineering.” 

The supposedly immutable laws underlying the quants’ models, 
however, didn’t work out, and the complex models turned out to 
have hidden risks rather than protecting against them, all at a ter- 
rible cost. Those risks, concealed and maybe even encouraged by 
the models, have led to hundreds of billions of dollars in losses to 
investors and taxpayers, to a global recession imposing trillions of 
dollars in losses to the world economy and immeasurable monetary 
and human costs. People around the world are losing their homes, 
their jobs, their dignity and their hope. 

Taxpayers here and around the world are shouldering the burden 
arising from financial firms’ miscalculation of risk, poor judgment, 
excessive bonuses and general profligate behavior. It is for this rea- 
son that the Subcommittee is directing our attention today to the 
intersection of quantitative analysis, economics and regulation. The 
Value-at-Risk model, or VaR, stands squarely at the intersection of 
quantitative analysis, economics and regulation. It is the most 
prominent risk model used by major financial institutions. The VaR 
is designed to provide an answer to the question, “What is the po- 
tential loss that could be faced within a limited, specified time to 
the value of an asset?” 
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The highest probability is that tomorrow’s value will be the same 
as today’s. The next highest probability is that there will be a 
small movement in value up or down, and so on. The more radical 
the movement in value, the lower the probability that it will hap- 
pen. In other words, the danger to a financial firm or the commu- 
nity comes at the extreme margins of the VaR distribution curve, 
in the tails of the distribution. As a map to day-to-day behavior, 
the VaR is probably pretty accurate for normal times, just as teams 
favored by odds makers usually win. But just as long shots some- 
times come home, just as underdogs do sometimes win, asset bub- 
bles or other non-normal market conditions also occur, and the VaR 
is unlikely to capture the risks and dangers. The VaR also cannot 
tell you when you have moved into non-normal market conditions. 

While the VaR was originally designed for financial institutions’ 
in-house use to evaluate short-term risk in their trading books, it 
has been given a key role in determining capital requirements for 
large banks under a major multilateral agreement, the Basel II Ac- 
cord, published in 2004. That same year, the U.S. Securities and 
Exchange Commission, the SEC, at the instigation of the five larg- 
est investment banks, adopted a capital reserve regime, applying 
Basel II standards to the Nation’s largest investment banks — a de- 
cision that opened the door to their over-leveraging and liquidity 
problems. Three of the institutions that asked the SEC for this 
change in rules — Bear Stearns, Merrill Lynch, Lehman Brothers — 
no longer exist. At the time, those financial institutions assured 
regulators that the VaR would reflect the level of risk they were 
taking on, and that a low VaR justified lower capital requirements. 
The result was exactly what the investment banks asked for: lower 
capital requirements that allowed them to invest in even more 
risky financial instruments all justified with risk models that as- 
sured regulators that there was nothing to worry about. What 
could possibly go wrong? 

In light of the VaR’s prominent role in the financial crisis, this 
subcommittee is examining that role and the role of related risk- 
measurement methods. From a policy perspective, the most impor- 
tant immediate question is how regulators use VaR numbers and 
other such models designed by regulated institutions, and whether 
they are an appropriate guide to setting capital reserve require- 
ments. But, beyond that, we must also ask whether the scientific 
and technical capabilities that led us into the current crisis should 
be applied to prevent future catastrophic events. Can mathematics, 
statistics and economics produce longer-range models, more reliable 
models, that could give us early warning when our financial system 
is headed for trouble? Or are such models inevitably going to be 
abused to hide risk-taking and encourage gambling by firms whose 
failures can throw the whole world into a recession, as they have 
in the last couple of years? If models cannot be a useful guide for 
regulation, should we just abandon the approach, or simply in- 
crease reserves, which will reduce profits and perhaps reduce some 
useful economic conduct in the short run, but protect taxpayers and 
the world economy in the long run? 

Those are big questions, but the stakes for taxpayers and inves- 
tors and the world economy justify some effort to get at some an- 
swers. 
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I now recognize Dr. Broun for his opening statement. 

[The prepared statement of Chairman Miller follows:] 

Prepared Statement of Chairman Brad Miller 

Economics has not been known in the past for mathematical precision. Harry Tru- 
man said he wanted a one-handed economist because he was frustrated with econo- 
mists who equivocated by saying “on the one hand ... on the other hand.” George 
Bernard Shaw said that if all the world’s economists were laid end to end, they still 
wouldn’t reach a conclusion. And apparently no one knows who first observed that 
economics was the only field in which two people can share a Nobel Prize for reach- 
ing exactly the opposite conclusion. 

But in the last 15 or 20 years, math and physics Ph.D.s from academia and the 
laboratory have entered the financial sector. Quantitative analysts, or “quants,” di- 
rected their mathematical and statistical skills to financial forecasts at a time when 
global financial markets were becoming more interdependent than ever before. 

The quants conceived such financial instruments as collaterized debt obligations, 
or “CDOs,” and credit default swaps, or “CDSs,” that would never have existed with- 
out them and their computers. They developed strategies for trading those instru- 
ments even in the absence of any underlying security or any real market. They con- 
structed risk models that convinced their less scientifically and technologically adept 
bosses that their instruments and strategies were infallibly safe. And their bosses 
spread faith in the quants’ models to regulators, who agreed to apply them to estab- 
lish capital reserve requirements that were supposed to guarantee the soundness of 
financial institutions against adverse events. It almost seemed like economic models 
had brought the precision of the laws of physics to financial risk management. Engi- 
neering schools even offered courses in “financial engineering.” 

The supposedly immutable “laws” underlying the quants’ models didn’t work, and 
the complex models turn out to have hidden risks rather than protected against 
them, all at a terrible cost. Those risks — concealed and maybe even encouraged by 
the models — have led to hundreds of billions of dollars in losses to investors and the 
taxpayers, to a global recession imposing trillions of dollars in losses to the world 
economy and immeasurable monetary and human costs. People around the world 
are losing their jobs, their homes, their dignity and their hope. 

Taxpayers here and around the world are shouldering the burden arising from fi- 
nancial firms’ miscalculation of risk, poor judgment, excessive bonuses and prof- 
ligate behavior. It is for this reason that the Subcommittee has chosen to direct its 
attention today to that intersection of quantitative analysis, economics, and regula- 
tion. The “Value-at-Risk” model, or “VaR” stands squarely at the center of this 
intersection as the most prominent risk model used by major financial institutions. 
The VaR is designed to provide an answer to the question, “What is the potential 
loss that could be faced within a limited, specified time to the value of an asset?” 

The highest probability is that tomorrow’s value will be the same as today’s; the 
next highest probability is of a very small movement in value up or down, and so 
on. The more radical the movement in value, the lower the probability of its occur- 
rence. In other words, the danger to the financial firm or the community comes at 
the extreme margins of the VaR distribution curve, in the “tails” of the distribution. 
As a map to day-to-day behavior, the VaR is probably pretty accurate for normal 
times, just as teams favored by odds makers usually win. But just as long shots 
sometimes come home, asset bubbles or other “non-normal” market conditions also 
occur, and the VaR is unlikely to capture the risks and dangers. The VaR also can- 
not tell you when you have moved into “non-normal” market conditions. 

While the VaR was originally designed for financial institutions’ to use in-house 
to evaluate short-term risk in their trading books, it was given a key role in deter- 
mining capital requirements for large banks under a major multilateral agreement, 
the Basel II Accord, published in 2004. That same year, the U.S. Securities and Ex- 
change Commission, at the instigation of the five largest investment banks, adopted 
a capital reserve regime applying Basel II standards to the Nation’s largest invest- 
ment banks, a decision that opened the door to their over-leveraging and liquidity 
problems. Three of the institutions that asked the SEC for this change in rules — 
Bear Stearns, Merrill Lynch, Lehman Brothers — no longer exist. At the time, those 
financial institutions assured regulators that the VaR would reflect the level of risk 
they were taking on, and that a low VaR justified lower reserve requirements. The 
result was exactly what the investment banks asked for; lower capital reserve re- 
quirements that allowed them to invest in even more risky financial instruments 
all justified with risk models that assured regulators that there was nothing to 
worry about. 
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In light of the VaR’s prominent role in the financial crisis, this Subcommittee is 
examining that role and the role of related risk-measurement methods. From a pol- 
icy perspective, the most important immediate question is how regulators use VaR 
numbers and other such models devised by regulated institutions and whether they 
are an appropriate guide to setting capital reserve requirements. But, beyond that, 
we must also ask whether the scientific and technical capabilities that helped lead 
us into the current crisis should be applied to prevent future catastrophic events. 
Can mathematics, statistics, and economics produce longer-range models — models 
that could give us early warning of when our complex financial system is heading 
for trouble? Or are such models inevitably going to be abused to hide risk-taking 
and encourage excessive gambling by firms whose failures can throw the whole 
world into a recession? If models cannot be a useful guide for regulation, should we 
just abandon this approach and simply increase reserves, reducing profits and per- 
haps some useful economic conduct in the short run, but protecting taxpayers and 
the world economy in the long run? 

These are big questions, but the stakes for taxpayers and investors and the world 
economy justify the effort to get at some answers. 

I now recognize Mr. Broun for his opening statement. 

Mr. Broun. Thank you, Mr. Chairman. Let me welcome the wit- 
nesses here today and thank them for appearing. Today’s hearing 
on financial modeling continues this committee’s work on the role 
of science in finance and economics. 

As I pointed out in our previous hearing in May, for the last sev- 
eral years Wall Street has increasingly leveraged mathematics, 
physics and science to better inform their decisions. Even before 
Value-at-Risk was developed to characterize risk, bankers and 
economists were looking for a silver bullet to help them to beat the 
market. 

Despite the pursuit of a scientific panacea for financial decisions, 
models are simply tools employed by decision-makers and risk 
managers. They add another layer of insight but are not crystal 
balls. Leveraging a position too heavily or assuming future sol- 
vency based on modeling data alone is hazardous, to say the least. 
Conversely, it stands to reason that if we could accurately predict 
markets, then both losses and profits would be limited since there 
would be very little risk involved. 

Modeling is a subject this committee has addressed several times 
in the past, whether it is in regard to climate change, chemical ex- 
posures, pandemics, determining spacecraft survivability or at- 
tempting to value complex financial instruments. Models are only 
as good as the data and assumptions that go into them. Ultimately 
decisions have to be made based on a number of variables which 
should include scientific models but certainly not exclusively. As 
witnesses in our previous hearing stated, “Science describes, it does 
not prescribe.” No model will ever relieve a banker, trader or risk 
manager of the responsibility to make difficult decisions and hedge 
inevitable uncertainly. 

This committee struggles enough with the complexities of mod- 
eling, risk assessment and risk management regarding physical 
sciences. Attempting to adapt those concepts to economics and fi- 
nance is even more complex. Appreciating this complexity and un- 
derstanding the limitations and intended purpose of financial mod- 
els is just as important as what the models tell you. 

We have two esteemed panels of witnesses here today who will 
discuss appropriate roles and limitations of models such as VaR. 
They will explain how these models are used and shed some light 
on what role they may have played in the recent economic crisis. 



10 


I look forward to you all’s testimony and I yield back my time. 
Thank you, Mr. Chairman. 

[The prepared statement of Mr. Broun follows:] 

Prepared Statement of Representative Paul C. Broun 

Thank you Mr. Chairman. 

Let me welcome the witnesses here today and thank them for appearing. 

Today’s hearing on Financial Modeling continues this committee’s work on the 
role of science in finance and economics. 

As I pointed out at our previous hearing in May, over the last 30 years Wall 
Street has increasingly leveraged mathematics, physics, and science to better inform 
their decisions. Even before Value-at-Risk (VaR) was developed to characterize risk, 
bankers and economists were looking for a silver bullet to help them beat the mar- 
ket. 

Despite the pursuit of a scientific panacea for financial decisions, models are sim- 
ply tools employed by decision-makers and risk managers. They add another layer 
of insight, but are not crystal balls. Leveraging a position too heavily or assuming 
future solvency based on modeling data alone is hazardous to say the least. Con- 
versely, it stands to reason that if we could accurately predict markets, then both 
losses and profits would be limited since there would be very little risk involved. 

Modeling is a theme this committee has addressed several times in the past. 
Whether it is in regard to climate change, chemical exposures, pandemics, deter- 
mining spacecraft survivability, or attempting to value complex financial instru- 
ments, models are only as good as the data and assumptions that go into them. Ulti- 
mately, decisions have to be made based on a number of variables which should in- 
clude scientific models, but certainly not exclusively. As a witness at a previous 
hearing stated, “science describes, it does not prescribe.” No model will ever relieve 
a banker, trader, or risk manager of the responsibility to make difficult decisions 
and hedge for inevitable uncertainty. 

This committee struggles enough with the complexities of modeling, risk assess- 
ment, and risk management regarding physical sciences. Attempting to adapt those 
concepts to economics and finance is even more complex. Appreciating this com- 
plexity, and understanding the limitations and intended purpose of financial models 
is just as important as what the models tell you. 

We have two esteemed panels of witnesses here today who will discuss the appro- 
priate roles and limitations of models such as VaR. They will explain how these 
models are used and shed some light on what role they may have played in the re- 
cent economic crisis. I look forward to their testimony and yield back my time. 

Thank you. 

Chairman Miller. Thank you, Dr. Broun. 

I now ask unanimous consent that all additional opening state- 
ments submitted by Members be included in the record. Without 
objection, that is so ordered. 


Panel I: 

We do have an outstanding group of witnesses today. I know that 
Chairmen at hearings always say that but it is certainly true. This 
time I mean it. On our first panel, we have two very well known 
and respected authors whose books and other writings warned 
against many of the practices of the financial industry that re- 
sulted in the current economic meltdown. Both of them have years 
of experience on Wall Street. Dr. Nassim Taleb is the author of 
“Fooled by Randomness” and “The Black Swan.” After a career as 
a trader and fund manager, Dr. Taleb is now the Distinguished 
Professor of Risk Engineering at the Polytechnic Institute of New 
York University. And if you are one of that slice of the American 
population for whom Bloomberg and CNBC are your favorite TV 
channels, Dr. Taleb is a rock star. Dr. Taleb is joined by another 
rock star, Dr. Richard Bookstaber, who is the author of “A Demon 
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of Our Own Design: Markets, Hedge Funds and the Risk of Finan- 
cial Innovation.” Dr. Bookstaber has worked as a risk manager for 
Salomon Brothers, Morgan Stanley and Moore Capital Manage- 
ment. He also runs equity funds and he began on Wall Street de- 
signing derivative instruments. Does your mother know about 
that? 

As our witnesses should know, you will each have five minutes 
for your spoken testimony. Your written testimony will be included 
in the record for the hearing. When you all have completed your 
spoken testimony, we will begin with questions and each Member 
will have five minutes to question the panel. It is the practice of 
this subcommittee — it is an investigative and oversight sub- 
committee — to receive testimony under oath. As I pointed out to 
the panelists at our last hearing on economic issues, to prosecute 
a case for perjury, the prosecutor, the U.S. attorney would have to 
prove what the truth was, that you knew the truth and that you 
consciously departed from it. I think you can sleep easily without 
worrying about a prosecution for perjury, but we will ask you to 
take an oath. Do either of you have any objection to taking an 
oath? Okay. You also have the right to be represented by counsel. 
Do either of you have counsel here? If you would, please stand and 
raise your right hand. Do you swear to tell the truth and nothing 
but the truth? 

The record will reflect that both witnesses did take the oath. We 
will begin with Dr. Taleb. Dr. Taleb. 

STATEMENT OF DR. NASSIM N. TALEB, DISTINGUISHED PRO- 
FESSOR OF RISK ENGINEERING, POLYTECHNIC INSTITUTE 

OF NEW YORK UNIVERSITY; PRINCIPAL, UNIVERSA INVEST- 
MENTS L.P. 

Dr. Taleb. Mr. Chairman, Ranking Member, Members of the 
Committee, thank you for giving me this opportunity to testify on 
the risk measurement methods used by banks, particularly those 
concerned with the risks of VaR events. You know, Value-at-Risk 
is just a method. It is a very general method, not very precise 
method, that measures the risks of VaR events. For example, a 
standard daily Value-at-Risk tells you that if your VaR is a million, 
daily VaR is a million, you have — it is at one percent probability, 
that you have less than one percent chance of losing a million or 
more on a given day. There are of course a lot of variations around 
VaR. For me, they are equally defective. 

Thirteen years ago, I wrote that the VaR encourages misdirected 
people to take risks with shareholders’ and ultimately taxpayers’ 
money — that is, regular people’s money. I have been since begging 
for suspension of these measurements of tail risks. We just don’t 
understand tail events. And lot of people say, oh, let’s measure 
risks. My idea is very different. Let’s find what risks we can meas- 
ure and any other risks we should be taking instead of doing it the 
opposite way. We take a lot of risks and then we try to find some 
scientists who can confirm these methods, you know, that these 
risks we can measure and that these methods are sound. 

I have been begging, and actually I wrote that I would be on the 
witness stand 13 years ago, and today I am here. The banking sys- 
tem lost so far more than $4.3 trillion, according to the Inter- 
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national Monetary Fund — that is more than they ever made in the 
history of banking — on tail risks, measurements of rare events. 
Most of the losses of course were in the United States, and I am 
not counting the economic consequences. But this shouldn’t have 
happened. Data shows that banks routinely lose everything they 
made over a long period of time in one single blow-up. It happened 
in 1982 because of multi-center banks losing everything made in 
the history of multi-center banking, one single event, loans to Latin 
America. The same thing in variation happened in 1991, and of 
course now. And every time society bails them out. Bank risk tak- 
ers retain their bonuses and say oh, one fluke, all right, and we 
start again. This is an aberrant case of capitalism for the profit, 
and socialism for the losses. 

So I have five points associated with VaR that I will go over very 
quickly, and I will give my conclusion. Number one: these problems 
were obvious all along. This should not have happened. We knew 
about the defects of the VaR when it was introduced. A lot of trad- 
ers, a lot of my friends, everyone — I am not the only person ranting 
against VaR. A lot of people were ranting against it before. Nobody 
heard us. Regulators did not listen to anyone who knew what was 
going on, is my point number one. 

Point number two: VaR is ineffective. I guess I don’t need more 
evidence than the recent events to convince you. 

Point number three, and that to me is crucial. You have a graph 
that shows you the performance profile of someone making steady 
earnings for a long time and then losing back everything. You can 
see from that graph, figure one on page four, that this is a strategy 
that is pretty much pursued by the majority of people on Wall 
Street, by banks. They make steady income for a long time, and 
when they blow up, they say, well, you know, it was unexpected, 
it was a black swan. I wrote a book called “The Black Swan.” Un- 
fortunately, they used my book backwards. Oh, and it was unex- 
pected, highly unexpected. They keep their bonuses. They go on va- 
cation and here you have a regular person working very hard, a 
taxpayer, a taxi driver, a post office worker paying taxes to sub- 
sidize retrospectively, all right, bonuses made. For example, a 
former government official made $121 million in bonuses at 
Citibank. Okay. He keeps his bonuses. We retrospectively are pay- 
ing for that. That I said 13 years ago, and it keeps happening, and 
now we are still in the same situation. 

So number four, and that is another crucial point. VaR has side 
effects. It is not neutral. You give someone a number — it has been 
shown and shown repeatedly, if you give someone a number, he 
will act on that number even if you tell him that that number is 
random. We humans cannot be trusted with numbers. You don’t 
give someone the map of the Alps if he is on the Mount Ararat, all 
right, because he is going to act on that map. Even nothing is alot 
better, if it doesn’t work. This is my central point, the side effects 
of numerical precision given to people who do not need it. 

Number five: VaR-style quantitative risk management was be- 
hind leverage. We increased our leverage in society as we thought 
we thought we could measure risk. If you think you can measure 
your blow-up risk, you are going to borrow, you know. You have 
more overconfidence, also, as a side effect of measurement, and you 
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are going to borrow. Instead of, you know, taking equity from peo- 
ple, you borrow, so when you blow up, you owe that money. And 
of course, as was discussed in my paper, debt bubbles are very vi- 
cious. Equity bubbles are not very vicious. 

Conclusion: What should we be doing? Well, regulators should 
understand that finance is a complex system and complex systems 
have very clear characteristics, you know, and one of them is low 
levels of predictability, particularly of tail events. We have to 
worry — regulators should not encourage model error. My idea is to 
build a society that is resistant to expert mistakes. Regulators in- 
creased the dependence of society on expert mistakes and other 
things also in the Value-at-Risk, these AAA things. Okay. So we 
want to reduce that. We want to build a society that can sustain 
shocks because we are moving more and more into a world that de- 
livers very large-scale variables, and we know exactly how they af- 
fect us or we know with some precision how they affect us, and we 
know how to build shocks. So the job of regulators should be to 
lower the impact of model error, and this is reminiscent of medi- 
cine. You know, the FDA, they don’t let you bring any medicine 
without showing the side effects. Well, we should be doing the 
same thing in economic life. Thank you very much for this oppor- 
tunity. 

[The prepared statement of Dr. Taleb follows:] 

Prepared Statement of Nassim N. Taleb 

Report on the Risks of Financial Modeling, 

VaR and the Economic Breakdown 

INTRODUCTION 

Mr. Chairman, Ranking Member, Members of the Committee, thank you for giv- 
ing me the opportunity to testify on the risk measurement methods used by banks, 
particularly those concerned with blowup risk, estimates of probabilities of losses 
from extreme events (“tail risks”), generally bundled under VaR. 1 

What is the VaR? It is simply a model that is supposed to project the expected 
extreme loss in an institution’s portfolio that can occur over a specific time frame 
at a specified level of confidence. Take an example. A standard daily VaR of $1 mil- 
lion at a one percent probability tells you that you have less than a one percent 
chance of losing $1 million or more on a given day. 2 There are many modifications 
around VaR, “conditional VaR,” 3 so my discussion concerns all quantitative (and 
probabilistic) methods concerned with losses associated with rare events. Simply, 
there are limitations to our ability to measure the risks of extreme events. 

Thirteen years ago, I warned that “VaR encourages misdirected people to take 
risks with shareholders’, and ultimately taxpayers’ money.” I have since been beg- 
ging for the suspension of these measurements of tail risks. But this came a bit late. 
For the banking system has lost so far, according to the International Monetary 
Fund, in excess of four trillion dollars directly as a result of faulty risk manage- 
ment. Most of the losses were in the U.S. and will be directly borne by taxpayers. 
These losses do not include the other costs of the economic crisis. 


1 The author thanks Daniel Kahneman, Pablo Triana, and Eric Weinstein for helpful discus- 
sions. 

2 Although such definition of VaR is often presented as a "maximum” loss, it is technically 
not so in an open-ended exposure: since, conditional on losing more than $1 million, you may 
lose a lot more, say $5 million. 

3 Data shows that methods meant to improve the standard VaR, like "expected shortfall” or 
“conditional VaR” are equally defective with economic variables — past losses do not predict fu- 
ture losses. Stress testing is also suspicious because of the subjective nature of “reasonable 
stress” number — we tend to underestimate the magnitude of outliers. “Jumps” are not predict- 
able from past jumps. See Taleb, N.N. (in press) “Errors, robustness, and the fourth quadrant,” 
International Journal of Forecasting (2009). 
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Data shows that banks routinely lose everything earned in their past history in 
single blowups — this happened in 1982, 1991, and, of course now. Every time society 
bails them out — while bank risk-takers retain their past bonuses and start the game 
afresh. This is an aberrant case of capitalism for the profits and socialism for the 
losses. 

MAIN PROBLEMS ASSOCIATED WITH VAR-STYLE RISK MEASUREMENT 

1. These problems have been obvious all along 

My first encounter with the VaR was as a derivatives trader in the early 1990s 
when it was first introduced. I saw its underestimation of the risks of a portfolio 
by a factor of 100 — you set up your book to lose no more than $100,000 and you 
take a $10,000,000 hit. Worse, there was no way to get a handle on how much its 
underestimation could be. 

Using VaR after the crash of 1987 proved strangely gullible. But the fact that its 
use was not suspended after the many subsequent major events, such as the Long- 
Term Capital Management blowup in 1998, requires some explanation. Further- 
more, regulators started promoting VaR (Basel 2) just as evidence was mounting 
against it. 4 

2. VaR is ineffective and lacks in robustness 

Alas, we cannot “measure” the risk of future rare events like we measure the tem- 
perature. By robustness, I mean that the measure does not change much if you 
change the model, technique, or theory. Indeed risk estimation has nothing to do 
with the notion of measure. And the rarer the event, the harder it is to compute 
its probability — yet the rarer the event, the larger the consequences. 5 

Furthermore, the type of randomness we have with economic variables does not 
have a well-tractable, well-known structure, and can deliver vastly large events — 
and we are unable to get a handle on “how large.” Conventional statistics, derived 
on a different class of variables, fail us here. 6 ' 7 G5, 8 

3. VaR encourages “low volatility, high blowup” risk taking which can be 
gamed by the Wall Street bonus structure 

Figure 1-A typical “blow-up” strategy with hidden risks: appearance of low vola- 
tility, with a high risk of blowup. The trader makes 11 bonuses, with no subsequent 
“clawback” as losses are borne by shareholders, then taxpayers. This is the profile 
for banks (losses in 1982,1991, and 2008) and many hedge funds. VaR encourages 
such types of risk taking. 


4 My recollection is that the VaR was not initially taken seriously by traders and managers. 
It took a long time for the practice to spread — and it was only after regulators got involved that 
it became widespread. 

5 See Taleb, N.N. and Pilpel, A. (2007) Epistemology and Risk Management, Risk and Regula- 
tion, 13. 

6 We are in the worst type of complex system characterized by high interdependence, low pre- 
dictability, and vulnerability to extreme events. See N.N. Taleb, The Black Swan , Random 
House, 2007. 

7 There are other problems. 1) VaR does not replicate out of sample — the past almost never 
predicts subsequent blowups, (see data in the Fourth Quadrant). 2) A decrease in VaR does not 
mean decrease in risks; often quite the opposite holds, which allows the measure to be gamed. 

8 The roots of VaR come from modern financial theory (Markowitz, Sharpe, Miller, Merton, 
Scholes) which, in spite of its patent lack of scientific validity, continues to be taught in business 
schools. See Taleb, N.N., (2000), The Black Swan: The Impact of the Highly Improbable, Random 
House. 
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Figure 1- A typical "blow-up" strategy with hidden risks: appearance of low volatility, with a 
high risk of blowup. The trader makes 11 bonuses, with no subsequent " clawback " as losses 
are borne by shareholders, then taxpayers. This is the profile for banks (losses in 1982, 1991, 
and 2008) and many hedge funds. VaR encourages such types of risk taking. 


I have shown that operators like to engage in a “blow-up” strategy, ( switching 
risks from visible to hidden), which consists in producing steady profits for a long 
time, collecting bonuses, then losing everything in a single blowup. 9 Such trades pay 
extremely well for the trader — but not for society. For instance, a member of 
Citicorp’s executive committee (and former government official) collected $120 mil- 
lion of bonuses over the years of hidden risks before the blowup; regular taxpayers 
are financing him retrospectively. 

Blowup risks kept increasing over the past few years, while the appearance of sta- 
bility has increased. 10 

4. Var has severe side effects (anchoring) 

Many people favor the adjunct application of VaR on grounds that it is “not harm- 
ful,” using arguments like “we are aware of its defects.” VaR has side effects of in- 
creasing risk-taking, even by those who know that it is not reliable. We have ample 
evidence of so called “anchoring” 11 in the calibration of decisions. Information, even 
when it is known to be sterile, increases overconfidence. 

5. VaR-style quantitative risk measurement is the engine behind leverage, 
the main cause of the current crisis 

Leverage 12 is a direct result of underestimation of the risks of extreme events — 
and the illusion that these risks are measurable. Someone more careful (or realistic) 
would issue equity. 

April 28, 2004 was a very sad day, when the SEC, at the instigation of the invest- 
ment banks, initiated the abandonment of hard (i.e., robust) risk measures like le- 
verage, in favor of more model-based probabilistic, and fragile, ones. 


9 Taleb, N.N. (2004) "Bleed or Blowup: What Does Empirical Psychology Tell Us About the 
Preference For Negative Skewness?,” Journal of Behavioral Finance, 5. 

10 Even Chairman Bernanke was fooled by the apparent stability as he pronounced it the 
“great moderation.” 

11 Numerous experiments provide evidence that professionals are significantly influenced by 
numbers that they know to be irrelevant to their decision, like writing down the last four digits 
of one’s social security number before making a numerical estimate of potential market moves. 
German judges rolling dice before sentencing showed an increase of 50 percent in the length 
of the sentence when the dice show a high number, without being conscious of it. See Birte 
Englich and Thomas Mussweiler, “Sentencing under Uncertainty: Anchoring Effects in the 
Courtroom,” Journal of Applied Social Psychology, Vol. 31, No. 7 (2001), pp. 1535—1551; Birte 
Englich, Thomas Mussweiler, and Fritz Strack, “Playing Dice with Criminal Sentences: the In- 
fluence of Irrelevant Anchors on Experts’ Judicial Decision Making,” Personality and Social Psy- 
chology Bulletin, Vol. 32, No. 2 (Feb. 2006), pp. 188-200. 

12 There is a large difference between equity and credit bubbles. Equity bubbles are benign. 
We went through an equity bubble in 2000, without major problems. 

Some credit can be benign. Credit that facilitates trade and economic transactions and fi- 
nances conservative house-ownership does not have the same risk properties as credit for specu- 
lative reasons resulting from overconfidence. 
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CONCLUSION: WHAT REGULATORY STRUCTURE DO WE NEED? 

Regulators should understand that financial markets are a complex system and 
work on increasing the robustness in it, by preventing “too big to fail” situations, 
favoring diversity in risk taking, allowing entities to absorb large shocks, and reduc- 
ing the effect of model error (see “Ten Points for a Black Swan Robust Society,” in 
Appendix II). This implies reliance on “hard,” non-probabilistic measures rather 
than more error-prone ones. For instance “leverage” is a robust measures (like the 
temperature, it does not change with your model), while VaR is not. 

Furthermore, we need to examine the toxicity of models; financial regulators 
should have the same test as the Food and Drug Administration does. The promoter 
of the probability model must be able to show that no one will be harmed even if 
the event is rare. Alas, the history of medicine shows translational gaps, the lag be- 
tween the discovery of harm and suspension of harmful practice, lasting up to 200 
years in pre-modern medicine. 13 Unfortunately, economics resemble pre-modern 
medicine. 14 But we cannot afford to wait 200 years to find out that the medicine 
is far worse than the disease. We cannot afford to wait even months. 

APPENDIX I: 


AUTHOR’S WARNINGS, 1996-2007 

1996-1997 

VaR is charlatanism because it tries to estimate something that is scientifically 
impossible to estimate, namely the risk of rare events. It gives people a misleading 
sense of precision. ( Derivatives Strategy, citing from Dynamic Hedging) 

VaR encourages misdirected people to take risks with shareholders’, and ulti- 
mately taxpayers’ money. ( Derivatives Strategy ) 

2003 

Fannie Mae’s models (for calibrating to the risks of rare events) are pseudo- 
science. ( New York Times — Alex Berenson’s article on FNMA) 

“What happened to LTCM will look like a picnic compared to what should happen 
to you.” ( Lecture , Women in Hedge Funds Association, cited in Hedge World) 

2007 

Fannie Mae, when I look at its risks, seems to be sitting on a barrel of dynamite, 
vulnerable to the slightest hiccup. But not to worry: their large staff of scientists 
deems these events “unlikely.” ( The Black Swan) 

Banks are now more vulnerable to the Black Swan than ever before with “sci- 
entists” among their staff taking care of exposures. The giant firm, J.P. Morgan, put 
the entire world at risk by introducing in the nineties RiskMetrics, a phony method 
aiming at managing people’s risks. A related method called ‘Value-at-Risk,” which 
relies on the quantitative measurement of risk, has been spreading. ( The Black 
Swan) 


13 “When William Harvey demonstrated the mechanism of blood circulation in the 1620s, hu- 
moral theory and its related practices should have disappeared, because the anatomy and physi- 
ology on which it relied was incompatible with this picture of the organism. In fact, people con- 
tinued to refer to spirits and humors, and doctors continued to prescribe phlebotomies, enemas, 
and cataplasms, for centuries more — even when it was established in the mid-1800, most nota- 
bly by Louis Pasteur, that germs were the cause of disease.” Noga Arikha “Just Life in a Nut- 
shell: Humours as common sense,” in The Philosophical Forum Quarterly, XXXIX, 3. 

14 Most of the use of probabilistic methods lacking both mathematical and empirical justifica- 
tion can be attributed to the prestige given to modern finance by the various Nobel memorial 
prizes in economics. See P. Triana, 2009, Lecturing Birds on Flying: Can Mathematical Theories 
Destroy the Markets ?, J. Wiley. 
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APPENDIX II: 

TEN PRINCIPLES FOR A BLACK SWAN 
ROBUST WORLD 

(FINANCIAL TIMES, APRIL 8, 2009) 

1. What is fragile should break early while it is still small. Nothing should 
ever become too big to fail. Evolution in economic life helps those with the max- 
imum amount of hidden risks — and hence the most fragile — become the biggest. 

2. No socialization of losses and privatization of gains. Whatever may need 
to be bailed out should be nationalized; whatever does not need a bail-out should 
be free, small and risk-bearing. We have managed to combine the worst of cap- 
italism and socialism. In France in the 1980s, the socialists took over the banks. 
In the U.S. in the 2000s, the banks took over the government. This is surreal. 

3. People who were driving a school bus blindfolded (and crashed it) 
should never be given a new bus. The economics establishment (universities, 
regulators, central bankers, government officials, various organizations staffed with 
economists) lost its legitimacy with the failure of the system. It is irresponsible and 
foolish to put our trust in the ability of such experts to get us out of this mess. In- 
stead, find the smart people whose hands are clean. 

4. Do not let someone making an “incentive” bonus manage a nuclear 
plant — or your financial risks. Odds are he would cut every corner on safety to 
show “profits” while claiming to be “conservative.” Bonuses do not accommodate the 
hidden risks of blow-ups. It is the asymmetry of the bonus system that got us here. 
No incentives without disincentives: capitalism is about rewards and punishments, 
not just rewards. 

5. Counter-balance complexity with simplicity. Complexity from 
globalization and highly networked economic life needs to be countered by 
simplicity in financial products. The complex economy is already a form of lever- 
age: the leverage of efficiency. Such systems survive thanks to slack and redun- 
dancy; adding debt produces wild and dangerous gyrations and leaves no room for 
error. Capitalism cannot avoid fads and bubbles: equity bubbles (as in 2000) have 
proved to be mild; debt bubbles are vicious. 

6. Do not give children sticks of dynamite, even if they come with a warn- 
ing. Complex derivatives need to be banned because nobody understands them and 
few are rational enough to know it. Citizens must be protected from themselves, 
from bankers selling them “hedging” products, and from gullible regulators who lis- 
ten to economic theorists. 

7. Only Ponzi schemes should depend on confidence. Governments should 
never need to “restore confidence.” Cascading rumors are a product of complex 
systems. Governments cannot stop the rumors. Simply, we need to be in a position 
to shrug off rumors, be robust in the face of them. 

8. Do not give an addict more drugs if he has withdrawal pains. Using le- 
verage to cure the problems of too much leverage is not homeopathy, it is denial. 
The debt crisis is not a temporary problem, it is a structural one. We need rehab. 

9. Citizens should not depend on financial assets or fallible “expert” ad- 
vice for their retirement. Economic life should be definancialized. We should 
learn not to use markets as storehouses of value: they do not harbor the certainties 
that normal citizens require. Citizens should experience anxiety about their own 
businesses (which they control), not their investments (which they do not control). 

10. Make an omelet with the broken eggs. Finally, this crisis cannot be fixed 
with makeshift repairs, no more than a boat with a rotten hull can be fixed with 
ad hoc patches. We need to rebuild the hull with new (stronger) materials; we will 
have to remake the system before it does so itself. Let us move voluntarily into Cap- 
italism 2.0 by helping what needs to be broken break on its own, converting debt 
into equity, marginalizing the economics and business school establishments, shut- 
ting down the “Nobel” in economics, banning leveraged buy-outs, putting bankers 
where they belong, clawing back the bonuses of those who got us here, and teaching 
people to navigate a world with fewer certainties. 

Then we will see an economic life closer to our biological environment: smaller 
companies, richer ecology, no leverage. A world in which entrepreneurs, not bank- 
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ers, take the risks, and companies are born and die every day without making the 
news. 

In other words, a place more resistant to black swans. 
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'conservative' Bonuses do not accommodate the hidden risks of blow-ups It is the asymmetry 
of the bonus system that got us here Mo incentives without diseicentives capealism m about 
rewards snd punishments, not just rewards 

5 Counter-balance compievty with stmphofy Complexly from globalisation and highly 
networked economic Ife needs to be countered by smpleey in financial piodudt The compter 
economy is already a form of leverage the leverage of efficiency Such systems survive thanks 
to slack and redundancy, adding debt produces wild and dangerous gyrations and leaves no 
room tor error Capitalitm cannot avoid fads and bubbles equity bubblet (as in 2000) have 
proved to be mrfd. debt bubbles are vicious 

6 Do not give children Nicks ol dynamo*, even rf they coma wlh a w amng Complex 
derrvakves noed 10 be banned because nobody understands them end few are retionai enough 
to know it Citizens must be protected from themselves, from bankers settng them ‘hedging* 
products, and from gullible regulators who Wen to economc theorists 

7 Only Pood schemas should depend on confidence Governments should never need to 
‘restore confidence* Cascading rumours aie a product of complei systems Governments 
cannot stop the rumoura Smpiy, we need to be in a position to shrug off rumours, be robust 
in the face of them 

8 Do not give an addict more dnjgsd he has withdrawal pains Using leverage to cure the 
problems of too much leverage is not homeopathy, it is demel The debt crisis is not a 
temporary problem, l is a structi*al one We need rehab 

9 Citizens should not depend on financial assets or fatobh 'expert' educe for that refwmenf 
Economic kfe should be defmanctaksed We should loam not to use markets as storehouses of 
value they do not harbour Vie certambea that noimal clizens require Citizens should 
eipenenco annety about then own businesses (which they control). not thee investments 
(which they do not control) 

10 U aha an omelette with the broken eggs Finaty. thrs chars cannot be fixed w-zh makeshift 
repairs, no more than a boat with a rotten hul can be fixed wth ad-hoc patches We need to 
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people to navigate a world wdh fewer certamtas 

Then we wdl see an economic life doser to our biological envuonment smaller companies, 
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companies are born and da every day without making the news 
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Statistical and applied probabilistic knowledge is the core of knowledge; 
statistics is what tells you if something is true, false, or merely anecdotal; it 
is the "logic of science"; it is the instrument of risk-taking; it is the applied 
tools of epistemology; you can't be a modern intellectual and not think 
probabilistically— but... let's not be suckers. The problem is much more 
complicated than it seems to the casual, mechanistic user who picked it up 
in graduate school. Statistics can foot you. In fact it is fooling your 
government right now. It can even bankrupt the system (let's face it: use 
of probabilistic methods for the estimation of risks did just blow up the 
banking system). 

THE FOURTH QUADRANT: A MAP OF THE LIMITS OF STATISTICS 

[ 9 . 15 . 08 ] 

By Nassim Nicholas Taleb 
An Edge Original Essay 



Introduction 

When Nassim Taleb talks about the limits of statistics, he becomes 
outraged. "My outrage," he says, "is aimed at the scientist-charlatan 
putting society at risk using statistical methods. This is similar to 
iatrogenics, the study of the doctor putting the patient at risk." As a 
researcher in probability, he has some credibility. In 2006, using FNMA and 
bank risk managers as his prime perpetrators, he wrote the following: 

The government-sponsored institution Fannie Mae, when I look 
at its risks, seems to be sitting on a barrel of dynamite, 
vulnerable to the slightest hiccup. But not to worry: their large 
staff of scientists deemed these events "unlikely." 

In the following Edge original essay, Taleb continues his 
examination of Black Swans, the highly improbable and 
unpredictable events that have massive impact. He claims 
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that those who are putting society at risk are "no true 
statisticians”, merely people using statistics either without 
understanding them, or in a self-serving manner. "The 
current subprime crisis did wonders to help me drill my 
point about the limits of statistically driven claims," he 
says. 

Taleb, looking at the cataclysmic situation facing financial institutions 
today, points out that "the banking system, betting against Black Swans, 
has lost over 1 Trillion dollars (so far), more than was ever made in the 
history of banking”. 

But, as he points out, there is also good news. 

We can identify where the danger zone is located, which I call 
"the fourth quadrant", and show it on a map with more or less 
clear boundaries. A map is a useful thing because you know 
where you are safe and where your knowledge is questionable. 

So I drew for the Edge readers a tableau showing the 
boundaries where statistics works well and where it is 
questionable or unreliable. Now once you identify where the 
danger zone is, where your knowledge is no longer valid, you 
can easily make some policy rules: how to conduct yourself in 
that fourth quadrant; what to avoid. 

— John Brockman 

NASSIM NICHOLAS TALEB, essayist and former mathematical trader, is 
Distinguished Professor of Risk Engineering at New York University's 
Polytechnic Institute. He is the author of Fooled by Randomness and the 
international bestseller The Black Swan. 

Nassim Taleb’s Edge Bio Pane 
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THE FOURTH QUADRANT: A MAP OF THE LIMITS OF STATISTICS 

Statistical and applied probabilistic knowledge is the core of knowledge; 
statistics is what tells you if something is true, false, or merely anecdotal; it 
is the "logic of science"; it is the instrument of risk-taking; it is the applied 
tools of epistemology; you can't be a modern intellectual and not think 
probabilistically— but... let's not be suckers. The problem is much more 
complicated than it seems to the casual, mechanistic user who picked it up 
in graduate school. Statistics can fool you. In fact it is fooling your 
government right now. It can even bankrupt the system (let's face it: use of 
probabilistic methods for the estimation of risks did just blow up the banking 
system). 

The current subprime crisis has been doing wonders for the reception of any 
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ideas about probability-driven claims in science, particularly in social 
science, economics, and "econometrics" (quantitative economics). Clearly, 
with current International Monetary Fund estimates of the costs of the 
2007-2008 subprime crisis, the banking system seems to have lost more on 
risk taking (from the failures of quantitative risk management) than every 
penny banks ever earned taking risks. But it was easy to see from the past 
that the pilot did not have the qualifications to fly the plane and was using 
the wrong navigation tools: The same happened in 1983 with money center 
banks losing cumulatively every penny ever made, and in 1991-1992 when 
the Savings and Loans industry became history. 

It appears that financial institutions earn money on transactions (say fees 
on your mother-in-law's checking account) and lose everything taking risks 
they don't understand. I want this to stop, and stop now— the current 
patching by the banking establishment worldwide is akin to using the same 
doctor to cure the patient when the doctor has a track record of 
systematically killing them. And this is not limited to banking— I generalize 
to an entire class of random variables that do not have the structure we 
thing they have, in which we can be suckers. 

And we are beyond suckers: not only, for socio-economic and other 
nonlinear, complicated variables, we are riding in a bus driven a blindfolded 
driver, but we refuse to acknowledge It in spite of the evidence, which to me 
is a pathological problem with academia. After 1998, when a "Nobel- 
crowned" collection of people (and the creme de la creme of the financial 
economics establishment) blew up Long Term Capital Management, a hedge 
fund, because the "scientific" methods they used misestimated the role of 
the rare event, such methodologies and such claims on understanding risks 
of rare events should have been discredited. Yet the Fed helped their bailout 
and exposure to rare events (and model error) patently increased 
exponentially (as we can see from banks' swelling portfolios of derivatives 
that we do not understand). 

Are we using models of uncertainty to produce certainties? 

This masquerade does not seem to come from statisticians— but from the 
commoditized, "me-too" users of the products. Professional statisticians can 
be remarkably introspective and self-critical. Recently, the American 
Statistical Association had a special panel session on the "black swan" 
concept at the annual Joint Statistical Meeting in Denver last August. They 
insistently made a distinction between the "statisticians" (those who deal 
with the subject itself and design the tools and methods) and those In other 
fields who pick up statistical tools from textbooks without really 
understanding them. For them it is a problem with statistical education and 
half-baked expertise. Alas, this category of blind users includes regulators 
and risk managers, whom I accuse of creating more risk than they reduce. 

So the good news is that we can identify where the danger zone is located, 
which I call "the fourth quadrant", and show it on a map with more or less 
clear boundaries. A map is a useful thing because you know where you are 
safe and where your knowledge is questionable. So I drew for the Edge 
readers a tableau showing the boundaries where statistics works well and 
where it is questionable or unreliable. Now once you identify where the 
danger zone is, where your knowledge is no longer valid, you can easily 
make some policy rules: how to conduct yourself in that fourth quadrant; 
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what to avoid. 

So the principal value of the map is that it allows for policy making. Indeed, 

I am moving on: my new project is about methods on how to domesticate 
the unknown, exploit randomness, figure out how to live in a world we don't 
understand very well. While most human thought (particularly since the 
enlightenment) has focused us on how to turn knowledge into decisions, my 
new mission is to build methods to turn lack of information, lack of 
understanding, and lack of "knowledge" into decisions— how, as we will see, 
not to be a "turkey”. 

This piece has a technical appendix that presents mathematical points and 
empirical evidence. (See link below.) It includes a battery of tests showing 
that no known conventional tool can allow us to make precise statistical 
claims in the Fourth Quadrant. While in the past I limited myself to citing 
research papers, and evidence compiled by others (a less risky trade), here 
I got hold of more than 20 million pieces of data (includes 98% of the 
corresponding macroeconomics values of transacted daily, weekly, and 
monthly variables for the last 40 years) and redid a systematic analysis that 
includes recent years. 


What Is Fundamentally Different About Real Life 

My anger with "empirical" claims in risk management does not come from 
research. It comes from spending twenty tense (but entertaining) years 
taking risky decisions in the real world managing portfolios of complex 
derivatives, with payoffs that depend on higher order statistical properties 
—and you quickly realize that a certain class of relationships that "look 
good" in research papers almost never replicate in real life (in spite of the 
papers making some claims with a "p" close to infallible). But that is not the 
main problem with research. 

For us the world is vastly simpler in some sense than the academy, vastly 
more complicated in another. So the central lesson from decision-making (as 
opposed to working with data on a computer or bickering about logical 
constructions) is the following: It is the exposure (or payoff) that creates the 
complexity —and the opportunities and dangers— not so much the 
knowledge ( i.e., statistical distribution, model representation, etc.). In some 
situations, you can be extremely wrong and be fine, in others you can be 
slightly wrong and explode. If you are leveraged, errors blow you up; if you 
are not, you can enjoy life. 

So knowledge (i.e., if some statement is "true" or "false") matters little, very 
little in many situations. In the real world, there are very few situations 
where what you do and your belief if some statement is true or false naively 
map into each other. Some decisions require vastly more caution than 
others— or highly more drastic confidence intervals. For instance you do not 
"need evidence" that the water is poisonous to not drink from it. You do not 
need "evidence" that a gun is loaded to avoid playing Russian roulette, or 
evidence that a thief a on the lookout to lock your door. You need evidence 
of safety— not evidence of lack of safety— a central asymmetry that affects 
us with rare events. This asymmetry in skepticism makes it easy to draw a 
map of danger spots. 
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The Dangers Of Bogus Math 

I start with my old crusade against "quants" (people like me who do 
mathematical work in finance), economists, and bank risk managers, my 
prime perpetrators of iatrogenic risks (the healer killing the patient). Why 
iatrogenic risks? Because, not only have economists been unable to prove 
that their models work, but no one managed to prove that the use of a 
model that does not work is neutral, that it does not increase blind risk 
taking, hence the accumulation of hidden risks. 


Anatomy of a Blowup 

I 
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Figure 1 My classical metaphor: A Turkey is fed for a 1000 
days— every days confirms to its statistical department that the 
human race cares about its welfare "with increased statistical 
significance”. On the 1001 st day, the turkey has a surprise. 


IndyMac's annual net income 

(In millions) 

$400 



Source: Bloomberg Sews 


Figure 2 The graph above shows the fate of close to 1000 
financial institutions (includes busts such as FNMA, Bear Stearns, 
Northern Rock, Lehman Brothers, etc.). The banking system 
(betting AGAINST rare events) just lost > 1 Trillion dollars (so 
far) on a single error, more than was ever earned in the history 
of banking. Yet bankers kept their previous bonuses and it looks 
like citizens have to foot the bills. And one Professor Ben 
Bernanke pronounced right before the blowup that we live in an 
era of stability and "great moderation" (he is now piloting a plane 
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and we all are passengers on it). 
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Figure 3 The graph shows the daily variations a derivatives 
portfolio exposed to U.K. interest rates between 1988 and 2008. 

Close to 99% of the variations, over the span of 20 years, will be 
represented in 1 single day— the day the European Monetary 
System collapsed. As I show in the appendix, this is typical with 
ANY socio-economic variable (commodity prices, currencies, 
inflation numbers, GDP, company performance, etc. ). No known 
econometric statistical method can capture the probability of the 
event with any remotely acceptable accuracy (except, of course, 
in hindsight, and "on paper"). Also note that this applies to 
surges on electricity grids and all manner of modern-day 
phenomena. 

Figures 1 and 2 show you the classical problem of the turkey making 
statements on the risks based on past history (mixed with some theorizing 
that happens to narrate well with the data). A friend of mine was sold a 
package of subprime loans (leveraged) on grounds that "30 years of history 
show that the trade is safe." He found the argument unassailable 
"empirically". And the unusual dominance of the rare event shown in Figure 
3 is not unique: it affects all macroeconomic data— if you look long enough 
almost all the contribution in some classes of variables will come from rare 
events (I looked in the appendix at 98% of trade-weighted data). 

Now let me tell you what worries me. Imagine that the Turkey can be the 
most powerful man in world economics, managing our economic fates. How? 
A then-Princeton economist called Ben Bernanke made a pronouncement in 
late 2004 about the "new moderation" in economic life: the world getting 
more and more stable— before becoming the Chairman of the Federal 
Reserve. Yet the system was getting riskier and riskier as we were 
turkey-style sitting on more and more barrels of dynamite— and Prof. 
Bernanke's predecessor the former Federal Reserve Chairman Alan 
Greenspan was systematically increasing the hidden risks in the system, 
making us all more vulnerable to blowups. 

By the "narrative fallacy” the turkey economics department will always 
manage to state, before thanksgivings that "we are in a new era of safety", 
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and back-it up with thorough and "rigorous" analysis. And Professor 
Bernanke indeed found plenty of economic explanations— what I call the 
narrative fallacy— with graphs, jargon, curves, the kind of facade- 
of-knowledge that you find In economics textbooks. (This is the find of glib, 
snake-oil facade of knowledge— even more dangerous because of the 
mathematics— that made me, before accepting the new position in NYU's 
engineering department, verify that there was not a single economist in the 
building. I have nothing against economists: you should let them entertain 
each others with their theories and elegant mathematics, and help keep 
college students inside buildings. But beware: they can be plain wrong, yet 
frame things in a way to make you feel stupid arguing with them. So make 
sure you do not give any of them risk-management responsibilities.) 


Bottom Line: The Map 

Things are made simple by the following. There are two distinct types of 
decisions, and two distinct classes of randomness. 

Decisions: The first type of decisions is simple, "binary”, i.e. you just care 
if something is true or false. Very true or very false does not matter. 
Someone is either pregnant or not pregnant. A statement is "true" or 
"false" with some confidence interval. (I call these MO as, more technically, 
they depend on the zero th moment, namely just on probability of events, 
and not their magnitude —you just care about "raw" probability). A 
biological experiment in the laboratory or a bet with a friend about the 
outcome of a soccer game belong to this category. 

The second type of decisions is more complex. You do not just care of the 
frequency— but of the impact as well, or, even more complex, some function 
of the impact. So there is another layer of uncertainty of impact. (I call 
these M1+, as they depend on higher moments of the distribution). When 
you invest you do not care how many times you make or lose, you care 
about the expectation: how many times you make or lose times the amount 
made or lost. 

Probability structures: There are two classes of probability 
domains— very distinct qualitatively and quantitatively. The first, 
thin-tailed: Mediocristan", the second, thick tailed Extremistan. Before I get 
into the details, take the literary distinction as follows: 

In Mediocristan, exceptions occur but don't carry large consequences. Add 
the heaviest person on the planet to a sample of 1000. The total weight 
would barely change. In Extremistan, exceptions can be everything (they 
will eventually, in time, represent everything). Add Bill Gates to your 
sample: the wealth will jump by a factor of >100,000. So, in Mediocristan, 
large deviations occur but they are not consequential— unlike Extremistan. 

Mediocristan corresponds to "random walk" style randomness that you tend 
to find in regular textbooks (and in popular books on randomness). 
Extremistan corresponds to a "random jump" one. The first kind I can call 
"Gaussian-Poisson", the second "fractal" or Mandelbrotian (after the works 
of the great Benoit Mandelbrot linking it to the geometry of nature). But 
note here an epistemological question: there is a category of "I don't know" 
that I also bundle in Extremistan for the sake of decision making— simply 
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because I don't know much about the probabilistic structure or the role of 
large events. 


The Map 

Now it lets see where the traps are: 

First Quadrant: Simple binary decisions, in Mediocristan: 
Statistics does wonders. These situations are, unfortunately, 
more common in academia, laboratories, and games than real 
life— what I call the "ludic fallacy". In other words, these are the 
situations In casinos, games, dice, and we tend to study them 
because we are successful in modeling them. 

Second Quadrant: Simple decisions, in Extremistan: some well 
known problem studied in the literature. Except of course that 
there are not many simple decisions in Extremistan. 

Third Quadrant: Complex decisions in Mediocristan: Statistical 
methods work surprisingly well. 

Fourth Quadrant: Complex decisions in Extremistan: Welcome 
to the Black Swan domain. Here is where your limits are. Do not 
base your decisions on statistically based claims. Or, 
alternatively, try to move your exposure type to make it third- 
quadrant style ("clipping tails"). 


APPLICATION 


Simple payoffs Complex payoffs 


DOMAIN 


Distribution 1 (“thin tailed") 


Extremely robust to Black 
Swans 


Quite robust to Black 
Swans 


Distribution 2 Quite robust to Black 

Swans 

("heavy" and/or unknown 
tails, no or unknown 
characteristic scale) 


LIMITS of Statistics - 
extreme fragility to Black 
Swans 
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The four quadrants. The South-East area (in orange) is where 
statistics and models fail us. 


Tableau Of Payoffs 
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Two Difficulties 

Let me refine the analysis. The passage from theory to the real world 

presents two distinct difficulties: "Inverse problems" and "pre-asymptotics". 

Inverse Problems. It is the greatest epistemological difficulty I 
know. In real life we do not observe probability distributions 
(not even in Soviet Russia, not even the French government). 

We just observe events. So we do not know the statistical 
properties— until, of course, after the fact. Given a set of 
observations, plenty of statistical distributions can correspond to 
the exact same realizations— each would extrapolate differently 
outside the set of events on which it was derived. The inverse 
problem is more acute when more theories, more distributions 
can fit a set a data. 

This inverse problem is compounded by the small sample 
properties of rare events as these will be naturally rare in a 
past sample. It is also acute in the presence of nonlinearities as 
the families of possible models/parametrization explode in 
numbers. 

Pre-asymptotics. Theories are, of course, bad, but they can be 
worse in some situations when they were derived in idealized 
situations, the asymptote, but are used outside the asymptote 
(its limit, say infinity or the infinitesimal). Some asymptotic 
properties do work well preasymptotically (Mediocristan), which 
is why casinos do well, but others do not, particularly when it 
comes to Extremistan. 

Most statistical education is based on these asymptotic, Platonic 
properties— yet we live in the real world that rarely resembles 
the asymptote. Furthermore, this compounds the ludic fallacy: 
most of what students of statistics do is assume a structure, 
typically with a known probability. Yet the problem we have is 
not so much making computations once you know the 
probabilities, but finding the true distribution. 


The Inverse Problem Of The Rare Events 

Let us start with the inverse problem of rare events and proceed with a 
simple, nonmathematical argument. In August 2007, The Wall Street 
Journal published a statement by one financial economist, expressing his 
surprise that financial markets experienced a string of events that "would 
happen once in 10,000 years". A portrait of the gentleman accompanying 
the article revealed that he was considerably younger than 10,000 years; it 
is therefore fair to assume that he was not drawing his inference from his 
own empirical experience (and not from history at large), but from some 
theoretical model that produces the risk of rare events, or what he 
perceived to be rare events. 


Alas, the rarer the event, the more theory you need (since we don't 
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observe it). So the rarer the event, the worse its inverse problem. And 
theories are fragile (just think of Doctor Bernanke). 

The tragedy is as follows. Suppose that you are deriving probabilities of 
future occurrences from the data, assuming (generously) that the past is 
representative of the future. Now, say that you estimate that an event 
happens every 1,000 days. You will need a lot more data than 1,000 days 
to ascertain its frequency, say 3,000 days. Now, what if the event happens 
once every 5,000 days? The estimation of this probability requires some 
larger number, 15,000 or more. The smaller the probability, the more 
observations you need, and the greater the estimation error for a set 
number of observations. Therefore, to estimate a rare event you need a 
sample that is larger and larger in inverse proportion to the occurrence of 
the event. 

If small probability events carry large impacts, and (at the same time) these 
small probability events are more difficult to compute from past data itself, 
then: our empirical knowledge about the potential contribution— or role— of 
rare events (probability x consequence) is inversely proportional to their 
impact. This is why we should worry in the fourth quadrant! 

For rare events, the confirmation bias (the tendency, Bernanke-styie, of 
finding samples that confirm your opinion, not those that disconfirm it) is 
very costly and very distorting. Why? Most of histories of Black Swan prone 
events is going to be Black Swan free! Most samples will not reveal the 
black swans— except after if you are hit with them, in which case you will 
not be in a position to discuss them. Indeed I show with 40 years of data 
that past Black Swans do not predict future Black Swans in socio-economic 
life. 
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Figure 4 The Confirmation Bias At Work. For left-tailed fat-tailed 
distributions, we do not see much of negative outcomes for 
surviving entities AND we have a small sample in the left tail. 

This is why we tend to see a better past for a certain class of 
time series than warranted. 


Fallacy Of The Single Event Probability 

Let us look at events in Mediocristan. In a developed country a newborn 
female is expected to die at around 79, according to insurance tables. When 
she reaches her 79th birthday, her life expectancy, assuming that she is in 
typical health, is another 10 years. At the age of 90, she should have 
another 4.7 years to go. So if you are told that a person is older than 100, 
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you can estimate that he is 102.5 and conditional on the person being older 
than 140 you can estimate that he is 140 plus a few minutes. The 
conditional expectation of additional life drops as a person gets older. 

In Extremistan things work differently and the conditional expectation of an 
increase in a random variable does not drop as the variable gets larger. In 
the real world, say with stock returns (and all economic variable), 
conditional on a loss being worse than the 5 units, to use a conventional 
unit of measure units, it will be around 8 units. Conditional that a move is 
more than 50 STD it should be around 80 units, and if we go all the way 
until the sample is depleted, the average move worse than 100 units is 250 
units! This extends all the way to areas in which we have sufficient sample. 

This tells us that there is "no typical" failure and "no typical" success. You 
may be able to predict the occurrence of a war, but you will not be able to 
gauge its effect! Conditional on a war killing more than 5 million people, it 
should kill around 10 (or more). Conditional on it killing more than 500 
million, it would kill a billion (or more, we don't know). You may correctly 
predict a skilled person getting "rich", but he can make a million, ten 
million, a billion, ten billion— there is no typical number. We have data, for 
instance, for predictions of drug sales, conditional on getting things right. 
Sales estimates are totally uncorrelated to actual sales— some drugs that 
were correctly predicted to be successful had their sales underestimated by 
up to 22 times! 

This absence of "typical" event in Extremistan is what makes prediction 
markets ludicrous, as they make events look binary. "A war" is meaningless: 
you need to estimate its damage— and no damage is typical. Many predicted 
that the First War would occur— but nobody predicted its magnitude. Of the 
reasons economics does not work is that the literature is almost completely 
blind to the point. 


A Simple Proof Of Unpredictability In The Fourth Quadrant 

I show elsewhere that if you don't know what a "typical" event is, fractal 
power laws are the most effective way to discuss the extremes 
mathematically. It does not mean that the real world generator is actually a 
power law— it means you don't understand the structure of the external 
events it delivers and need a tool of analysis so you do not become a turkey. 
Also, fractals simplify the mathematical discussions because all you need is 
play with one parameter (I call it "alpha”) and it increases or decreases the 
role of the rare event in the total properties. 

For instance, you move alpha from 2.3 to 2 in the publishing business, and 
the sales of books in excess of 1 million copies triple! Before meeting Benoit 
Mandelbrot, I used to play with combinations of scenarios with series of 
probabilities and series of payoffs filling spreadsheets with clumsy 
simulations; learning to use fractals made such analyses immediate. Now all 
I do is change the alpha and see what’s going on. 

Now the problem: Parametrizing a power law lends itself to monstrous 
estimation errors (I said that heavy tails have horrible inverse problems). 
Small changes in the "alpha" main parameter used by power laws leads to 
monstrously large effects in the tails. Monstrous. 
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And we don’t observe the "alpha. Figure 5 shows more than 40 thousand 
computations of the tail exponent "alpha" from different samples of different 
economic variables (data for which it is impossible to refute fractal power 
laws). We dearly have problems figuring it what the "alpha" is: our results 
are marred with errors. Clearly the mean absolute error is in excess of 1 
(i.e. between alpha=2 and alpha=3). Numerous papers in econophysics 
found an "average" alpha between 2 and 3— but if you process the >20 
million pieces of data analyzed in the literature, you find that the variations 
between single variables are extremely significant. 



Figure S— Estimation error in "alpha" from 40 thousand 
economic variables. I thank Pallop Angsupun for data. 

Now this mean error has massive consequences. Figure 6 shows the effect: 
the expected value of your losses in excess of a certain amount(called 
"shortfall") is multiplied by >10 from a small change In the "alpha" that is 
less than its mean error! These are the losses banks were talking about with 
confident precision! 


Alpha and remote events 



Figure 6— The value of the expected shortfall (expected losses in 
excess of a certain threshold) in response to changes in tail 
exponent "alpha". We can see it explode by an order of 
magnitude. 

What if the distribution is not a power law? This is a question I used to get 
once a day. Let me repeat it: my argument would not change— it would take 
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longer to phrase it. 

Many researchers, such as Philip Tetlock, have looked into the incapacity of 
social scientists in forecasting (economists, political scientists). It is thus 
evident that while the forecasters might be just "empty suits", the forecast 
errors are dominated by rare events, and we are limited in our ability to 
track them. The "wisdom of crowds" might work in the first three quadrant; 
but it certainly fails (and has failed) in the fourth. 


Living In The Fourth Quadrant 

Beware the Charlatan. When I was a quant-trader in complex derivatives, 
people mistaking my profession used to ask me for "stock tips" which put me 
in a state of rage: a charlatan is someone likely (statistically) to give you 
positive advice, of the "how to" variety. 

Go to a bookstore, and look at the business shelves: you will find plenty of 
books telling you how to make your first million, or your first quarter-billion, 
etc. You will not be likely to find a book on "how I failed in business and in 
life"— though the second type of advice is vastly more informational, and 
typically less charlatanic. Indeed, the only popular such finance book I found 
that was not quacky in nature— on how someone lost his fortune— was both 
self-published and out of print. Even in academia, there is little room for 
promotion by publishing negative results— though these, are vastly 
informational and less marred with statistical biases of the kind we call data 
snooping. So all I am saying is "what is it that we don't know", and my 
advice is what to avoid, no more. 

You can live longer if you avoid death, get better if you avoid bankruptcy, 
and become prosperous if you avoid blowups in the fourth quadrant. 

Now you would think that people would buy my arguments about lack of 
knowledge and accept unpredictability. But many kept asking me "now that 
you say that our measures are wrong, do you have anything better?" 

I used to give the same mathematical finance lectures for both graduate 
students and practitioners before giving up on academic students and grade- 
seekers. Students cannot understand the value of "this is what we don't 
know"— they think it is not information, that they are learning nothing. 
Practitioners on the other hand value it immensely. Likewise with 
statisticians: I never had a disagreement with statisticians (who build the 
field)— only with users of statistical methods. 

Spyros Makridakis and I are editors of a special issue of a decision science 
journal, The International Journal of Forecasting. The issue is about "What to 
do in an environment of low predictability”. We received tons of papers, but 
guess what? Very few addressed the point: they mostly focused on showing 
us that they predict better (on paper). This convinced me to engage in my 
new project: "how to live in a world we don’t understand". 

So for now I can produce phrenetic rules (in the Aristotelian sense of 
phronesis, decision-making wisdom). Here are a few, to conclude. 
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Phronetic Rules: What Is Wise To Do (Or Not Do) In The Fourth 
Quadrant 

1) Avoid Optimization, Learn to Love Redundancy. Psychologists tell us 
that getting rich does not bring happiness— if you spend it. But if you hide it 
under the mattress, you are less vulnerable to a black swan. Only fools 
(such as Banks) optimize, not realizing that a simple model error can blow 
through their capital (as it just did). In one day in August 2007, Goldman 
Sachs experienced 24 x the average daily transaction volume— would 29 
times have blown up the system? The only weak point I know of financial 
markets is their ability to drive people & companies to "efficiency" (to please 
a stock analyst's earnings target) against risks of extreme events. 

Indeed some systems tend to optimize— therefore become more fragile. 
Electricity grids for example optimize to the point of not coping with 
unexpected surges— Albert-Lazlo Barabasi warned us of the possibility of a 
NYC blackout like the one we had in August 2003. Quite prophetic, the 
fellow. Yet energy supply kept getting more and more efficient since. 
Commodity prices can double on a short burst in demand (oil, copper, 
wheat) —we no longer have any slack. Almost everyone who talks about 
"flat earth" does not realize that it is overoptimized to the point of maximal 
vulnerability. 

Biological systems— those that survived millions of years— include huge 
redundancies. lust consider why we like sexual encounters (so redundant to 
do it so often!). Historically populations tended to produced around 4-12 
children to get to the historical average of ~2 survivors to adulthood. 

Option-theoretic analysis: redundancy is like long an option. You certainly 
pay for it, but it may be necessary for survival. 

2) Avoid prediction of remote payoffs— though not necessarily ordinary 
ones. Payoffs from remote parts of the distribution are more difficult to 
predict than closer parts. 

A general principle is that, while in the first three quadrants you can use the 
best model you can find, this is dangerous in the fourth quadrant: no model 
should be better than just any model. 

3) Beware the "atypicality" of remote events. There is a sucker's 
method called "scenario analysis" and "stress testing"— usually based on the 
past (or some "make sense" theory). Yet I show in the appendix how past 
shortfalls that do not predict subsequent shortfalls. Likewise, "prediction 
markets" are for fools. They might work for a binary election, but not in the 
Fourth Quadrant. Recall the very definition of events is complicated: success 
might mean one million in the bank ...or five billions! 

4) Time. It takes much, much longer for a times series in the Fourth 
Quadrant to reveal its property. At the worst, we don’t know how long. Yet 
compensation for bank executives is done on a short term window, causing a 
mismatch between observation window and necessary window. They get rich 
in spite of negative returns. But we can have a pretty clear idea if the "Black 
Swan" can hit on the left (losses) or on the right (profits). 

The point can be used in climatic analysis. Things that have worked for a 
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long time are preferable— they are more likely to have reached their ergodic 
states. 


5) Beware Moral Hazard. Is optimal to make series of bonuses betting on 
hidden risks in the Fourth Quadrant, then blow up and write a thank you 
letter. Fannie Mae and Freddie Mac's Chairmen will in all likelihood keep 
their previous bonuses (as in all previous cases) and even get close to 15 
million of severance pay each. 

6) Metrics. Conventional metrics based on type 1 randomness don't work. 
Words like "standard deviation" are not stable and does not measure 
anything In the Fourth Quadrant. So does "linear regression” (the errors are 
in the fourth quadrant), "Sharpe ratio", Markowitz optimal portfolio, ANOVA 
shmnamova, Least square, etc. Literally anything mechanistically pulled out 
of a statistical textbook. 


My problem is that people can both accept the role of rare events, agree 
with me, and still use these metrics, which is leading me to test if this is a 
psychological disorder. 

The technical appendix shows why these metrics fail: they are based on 
"variance"/"standard deviation" and terms invented years ago when we had 
no computers. One way I can prove that anything linked to standard 
deviation is a facade of knowledge: There is a measure called Kurtosis that 
indicates departure from "Normality”. It is very, very unstable and marred 
with huge sampling error: 70-90% of the Kurtosis in Oil, SP500, Silver, UK 
interest rates, Nikkei, US deposit rates, sugar, and the dollar/yet currency 
rate come from 1 day In the past 40 years, reminiscent of figure 3. This 
means that no sample will ever deliver the true variance. It also tells us 
anyone using "variance” or "standard deviation" (or worse making models 
that make us take decisions based on it) in the fourth quadrant is 
incompetent. 

7) Where is the skewness? Clearly the Fourth Quadrant can present left 
or right skewness. If we suspect right-skewness, the true mean is more 
likely to be underestimated by measurement of past realizations, and the 
total potential is likewise poorly gauged. A biotech company (usually) faces 
positive uncertainty, a bank faces almost exclusively negative shocks. I call 
that in my new project "concave" or "convex” to model error. 


Unseen Positive Tail 



Concave lo model Convex lo model error 
Error 

Ex: Banks. Short Example: Long 

volatility, security options. Biotech. 

technology, 
entrepreneurship, cte. 
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8) Do not confuse absence of volatility with absence of risks. Recall 
how conventional metrics of using volatility as an indicator of stability has 
fooled Bernanke— as well as the banking system. 
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Figure 7 Random Walk— Characterized by volatility. You only 
find these in textbooks and in essays on probability by people 
who have never really taken decisions under uncertainty. 
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Figure 8 Random Jump process— It is not characterized by its 
volatility. Its exits the 80-120 range much less often, but its 
extremes are far more severe. Please tell Bernanke if you have 
the chance to meet him. 

9) Beware presentations of risk numbers. Not only we have 
mathematical problems, but risk perception is subjected to framing issues 
that are acute in the Fourth Quadrant. Dan Goldstein and I are running a 
program of experiments in the psychology of uncertainty and finding that 
the perception of rare events is subjected to severe framing distortions: 
people are aggressive with risks that hit them "once every thirty years" but 
not if they are told that the risk happens with a "3% a year" occurrence. 
Furthermore it appears that risk representations are not neutral: they cause 
risk taking even when they are known to be unreliable. 


Technical Appendix to "The Fourth Quadrant"— Click Here 
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Abstract 

The paper presents evidence that econometric techniques based on variance - L~ norm - are flawed and do not replicate. The 
result is un-computability of the role of tail events. The paper proposes a methodology to calibrate decisions to the degree (and 
computability) of forecast error. It classifies decision payoffs in two types: simple (true/falsc or binary) and complex (higher 
moments); and randomness into type- 1 (thin tails) and type-2 (true fat tails), and shows the errors for the estimation of small 
probability payoffs for type 2 randomness. The fourth quadrant is where payoffs arc complex with type-2 randomness. We 
propose solutions to mitigate the effect of the fourth quadrant, based on the nature of complex systems. 

© 2009 International Institute of Forecasters. Published by Elsevier B.V. All rights reserved. 
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1. Background and purpose 

It appears scandalous that, of the hundreds of 
thousands of professionals involved, including prime 
public institutions such as the World Bank, the 
International Monetary Fund, different governmental 
agencies and central banks, private institutions such as 
banks, insurance companies, and large corporations, 
and. finally, academic departments, only a few 
individuals considered the possibility of the total 
collapse of the banking system that started in 2007 
(and is still worsening at the time of writing), let alone 
the economic consequences of such breakdown. Not 
a single official forecast turned out to be close to the 
outcome experienced — even those issuing “warnings” 


E-mail address: nnttPfoolcdbyrandomncss.com. 


did not come close to the true gravity of the situation. 
A few warnings about the risks, such as Taleb (2007a) 
or the works of the economist Nouricl Roubini, 1 
went unheeded, often ridiculed. 2 Where did such 
sophistication go? in the face of miscalculations of 
such proportion, it would seem fitting to start an 
examination of the conventional forecasting methods 
for risky outcomes and assess their fragility — indeed, 
the size of the damage comes from confidence 
in forecasting and the mis-cstimation of potential 
forecast errors for a certain classes of variables and 
a certain type of exposures. However, this was not 

* “Dr. Doom”, New York Times, August 1 5. 2008. 

2 Note the irony that the ridicule of the warnings in Taleb (2007a) 
and other ideas came from the academic establishment, not from the 
popular press. 


0169-2070/S - see front matter © 2009 International Institute of Forecasters, Published by Elsevier B, V. All rights reserved, 
doi: 10. 10 l6/j.ijforccast.2009.05.027 
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the first time such events have happened — nor was it 
a “Black Swan" (when capitalized, an unpredictable 
outcome of high impact) to the observer who took 
a close look at the robustness and empirical validity 
of the methods used in economic forecasting and risk 
measurement. 

This examination, while grounded in economic 
data, generalizes to all decision-making under 
uncertainty in which there is a potential miscalculation 
of the risk of a consequential rare event. The problem 
of concern is the rare event, and the exposure to it, of 
the kind that can fool a decision maker into taking a 
certain course of action based on a misunderstanding 
of the risks involved. 

2. Introduction 

Forecasting is a serious professional and scientific 
endeavor with a certain purpose, namely to provide 
predictions to be used in formulating decisions, and 
taking actions. The forecast translates into a decision, 
and, accordingly, the uncertainty attached to the 
forecast, i.e., the error, needs to be endogenous to 
the decision itself. This holds particularly true of risk 
decisions. In other words, the use of the forecast needs 
to be determined — or modified — based on the 
estimated accuracy of the forecast. This in turn creates 
an interdependency about what we should or should 
not forecast — as some forecasts can be harmful to 
decision makers. 

Fig. 1 gives an example of harm coming from 
building risk management on the basis of extrapolative 
(usually highly technical) econometric methods, 
providing decision-makers with false confidence about 
the risks, and leaving society exposed to several 
trillions in losses that put capitalism on the verge of 
collapse. 

A key word here, fat tails , implies the outsized role 
in the total statistical properties played by one single 
observation — such as one massive loss coming after 
years of stable profits or one massive variation unseen 
in past data. 

- “Thin-tails" lead to ease in forecasting and 
tractability of the errors; 

- ‘Thick-tails” imply more difficulties in getting a 
handle on the forecast errors and the fragility of the 
forecast. 



Fig. 1. Indy Mac's annual income (in millions) between 1998 
and 2007. We can see fai tails at work. Tragic errots come from 
underestimating potential losses, with the best known cases being 
FNMA. Freddie Mac. Bear Stearns, Northern Rock, and Lehman 
Brothers, in addition to numerous hedge funds. 

Close to 1000 financial institutions have shut down 
in 2007 and 2008 from the underestimation of outsized 
market moves, with losses up to 3.6 trillion. 3 Had 
their managers been aware of the unreliability of the 
forecasting methods (which were already apparent in 
the data), they would have requested a different risk 
profile, with more robustness in risk management and 
smaller dependence on complex derivatives. 

2.1. The smoking gun 

We conducted a simple scientific examination 
of economic data, using a near-exhaustive set that 
includes 38 “tradable" variables 4 that allow for 
daily prices: major equity indices across the globe 
(US, Europe. Asia, Latin America), most metals 
(gold, silver), major interest rate securities, and main 
currencies — what we believe represents around 98% 
of tradable volume. 


3 Bloomberg, Feb 5. 2009. 

4 We selected a near-exhaustive set of economic data that includes 
'tradable" securities that allow for a future or a forward market: 
most equity indices across the globe, most metals, most interest 
rate securities, and most currencies. Wc collected all available 
traded futures data— what we believe represents around 98% of 
tradable volume. The reason we selected tradable data is because 
of the certainty of the practical aspect of a price on which one can 
transact: a nontradable currency price can lend itself to all manner 
of manipulation, More precisely we selected "continuously rolled” 
futures in winch the returns from holding a security are built-in. For 
instance, analyses of Dow Jones that fail to account for dividend 
payments or analyses of currencies that do not include interest rates 
provide a bias in the measurement of the mean and lughcr moments. 


Please cite this article in press as Taleb, N. N. Errors, robustness, and the fourth quadrant. International Journal of Forecasting < 2009 ). 
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Fig. 2. Tlie smoking gun: Maximum contribution to the fourth 
moment kuitosis coming from the largest observation in ~ 10,000 
(29-40 years of daily observations) for 43 economic variables. For 
tire Gaussian the number is expected to be ~0.006 for « = 10.000. 


We analyzed the properties of the logarithmic 
returns r t% tst — Log )’ where At can be 1 day, 
10 days, or 66 days (non-overlapping intervals). 5 

A conventional test of nonnormality used in the 
literature is the excess kurtosis over the normal 
distribution. Thus, we measured the fourth noncentral 
moment k(Ar) = the distributions and 

focused on the stability of the measurements. 

By examining Table 1 and Figs. 2 and 3, it appears 
that: 

(1) Economic variables (currency rates, financial 
assets, interest rates, commodities) are patently fat 


5 By convention wc use t = 1 as one business day. 


tailed — with no known exception. The literature 
(Bundt & Murphy, 2006) shows that this also 
applies to data not considered here, owing to a lack 
of daily changes, such as GDP, or inflation. 

(2) Conventional methods, not just those relying on 
a Gaussian distribution, but those based on least- 
square methods, or using variance as a measure of 
dispersion, are, according to the data, incapable 
of tracking the kind of “fat-tails” we see (more 
technically, in the L 2 norm, as will be discussed in 
Section 5). The reason is that most of the kurtosis 
is concentrated in a few observations, making 
it practically unknowable using conventional 
methods— see Fig. 2. Other tests in Section 5 
(the conditional expectation above a threshold) 
show further instability. This incapacitates least- 
square methods, linear regression, and similar 
tools, including risk management methods such 
as “Gaussian Copulas" that rely on correlations or 
any form of the product of random variables. 6, 7 - 8 


6 This should predict, for instance, the total failure in practice 
of the ARCH/GARCH methods (Engle. 1982). in spite of their 
successes in-sample. and in academic citations. as they are based 
on the behavior of squares. 

7 One counterintuive result is that sophisticated operators do not 
seem to be aware of the norm they are using, thus mis estimating 
volatility, see Goldstein and Taleb (2007). 

8 Practitioners have blamed the naive L ? reliance on the risk 
management of credit risk for the blowup of banks in the crisis 
that started in 2007. See Felix Salmon's "Recipe For Disaster. The 
Formula That Killed Wall Street" in Wired. 02/23/2009. 
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Fig. 3. A selection of the 12 most acute cases among the 43 economic variables. 
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Table I 

Fourth NoDcenir.il Moment at daily, 10 day. aod 66 day windows for the random variables. 



*(1) 

A' (10) 

A' (66) 

Max quaitk 

Years 

Australian DollatfUSD 

63 

33 

2.9 

0.12 

22 

Australia TB lOy 

7 3 

63 

33 

0.08 

25 

Australia TB 3y 

7 3 

54 

43 

0.06 

21 

Bean Oil 

53 

7.0 

4.9 

0.11 

47 

Bonds 30Y 

5.6 

4.7 

3.9 

0.02 

32 

Bcvcspa 

24.9 

5.0 

23 

037 

16 

British Pound/USD 

6.9 

74 

53 

0.05 

38 

CAC40 

63 

4.7 

36 

0.05 

20 

Canadian Dollar 

74 

4.1 

3.9 

0.06 

38 

Cocoa NY 

4.9 

4.0 

53 

0.04 

47 

Coffee NY 

10.7 

53 

53 

0.13 

37 

Copper 

64 

5.5 

43 

0.05 

48 

Com 

94 

8.0 

53 

0.18 

49 

Crude Oil 

29.0 

4.7 

5.1 

0.79 

26 

CT 

73 

43 

3.7 

035 

48 

DAX 

8.0 

63 

3.7 

03 

IS 

Euro Bund 

4.9 

33 

33 

0.06 

18 

EuroCurreocy/DEM previously 

53 

33 

23 

0.06 

38 

Eurodollar Depo 1M 

413 

28.0 

63 

031 

19 

Eurodollar Depo 3M 

21.1 

8.1 

73 

035 

28 

FTSE 

15 3 

274 

63 

034 

25 

Gold 

11.9 

143 

166 

0.04 

35 

Heating Oil 

20.0 

4.1 

44 

0.74 

31 

Hogs 

43 

46 

43 

0.05 

43 

Jakarta Stock Index 

403 

63 

43 

0.19 

16 

Japanese Gov Bonds 

173 

16.9 

43 

048 

24 

Live Cattle 

4 3 

4.9 

56 

0.04 

44 

Nasdaq Index 

114 

93 

53 

0.13 

21 

Natural Gas 

6.0 

3.9 

33 

0.06 

19 

Nikkei 

526 

4.0 

2.9 

0.72 

23 

Notes 5Y 

5.1 

33 

23 

0.06 

21 

Russia RTSI 

133 

6.0 

73 

0.13 

17 

Sliort Sterling 

8513 

93.0 

33 

0.75 

17 

Silver 

1603 

226 

103 

0.94 

46 

Snolkap 

6.1 

5.7 

63 

0.06 

17 

Soy Beans 

7.1 

83 

6.7 

0.17 

47 

Soy Meal 

8.9 

93 

83 

0.09 

48 

SpSOO 

383 

7.7 

5.1 

0.79 

56 

Sugar n 1 1 

94 

64 

33 

0.3 

48 

SwissFranc 

5.1 

33 

26 

0.05 

38 

TY10Y Notes 

5.9 

53 

4.9 

0.1 

27 

Wheal 

5.6 

6.0 

6.9 

0.02 

49 

Yen/USD 

9.7 

6.1 

23 

037 

38 


(3) There is no evidence of “convergence lo normal- 
ity" by aggregation, i.e., looking at the kurtosis of 
weekly or monthly changes. The “fatness" of the 
tails seems to be conserved under aggregation. 

Clearly, had decision-makers been aware of such 
facts, and such unreliability of conventional methods 


in tracking large deviations, fewer losses would have 
been incurred, as they would have reduced exposures 
in some areas rather than rely on more “sophisticated" 
methods. The financial system has been fragile, as this 
simple test shows, with the evidence staring at us all 
along. 
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2.2. The problem of large deviations 

2.2.1. The empirical problem of small probabilities 

The central problem addressed in this paper is that 

small probabilities are difficult to estimate empirically 
(since the sample set for these is small), with a 
greater error rate than that lor more frequent events. 
But since, in some domains, their effects can be 
consequential, the error concerning the contribution 
of small probabilities to the total moments of the 
distribution becomes disproportionately large. The 
problem has been dealt with by assuming a probability 
distribution and extrapolating into the tails — which 
brings model error into play. Yet, as we will discuss, 
model error plays a larger role with large deviations. 

2.2.2. Links to decision theory 

It is not necessary here to argue that a decision 
maker needs to use a full tableau of payoffs (rather 
than the simple one-dimensional average forecast) and 
that payoffs from decisions vary in their sensitivity to 
forecast errors. For instance, while it is acceptable to 
take a medicine that might be effective with a 5% error 
rate, but offers no side effects otlierwise. it is foolish 
to play Russian roulette with the knowledge that one 
should win with a 5 % error rate — indeed, standard 
theory of choice under uncertainty requires the use of 
full probability distributions, or at least a probability 
associated with every payoff. But so far this simple 
truism has not been integrated into the forecasting 
activity itself — as no classification has been made 
concerning the tractability and consequences of the 
errors. To put it simply, the mere separation between 
forecasting and decisions is lacking in both rigor and 
practicality, as it ruptures the link between forecast 
error and the quality of the decision. 

The extensive literature on decision theory and 
choices under uncertainty so far has limited itself to 
(1) assuming known probability distributions (except 
for a few exceptions in which this type of uncertainty 
has been called “ambiguity” 9 ), and (2) ignoring fat 
tails. This paper introduces a new structure of fat 
tails and classification of classes of randomness into 
the analysis, and focuses on the interrelation between 
errors and decisions. To establish a link between 


9 EUsberg's paradox. Ellsberg (1961); see also Ganlenfors and 

Sahlin ( 1982) and Ixni (1986). 


decision and quality of forecast, this analysis operates 
along two qualitative lines: qualitative differences 
between decisions along their vulnerability to error 
rates on one hand, and qualitative differences between 
two types of distributions of error rates. So there are 
two distinct types of decisions, and two distinct classes 
of randomness. 

This classification allows us to isolate situations 
in which forecasting needs to be suspended — or a 
revision of the decision or exposure may be necessary. 
What we call the “fourth quadrant" is the area in which 
both the magnitude of forecast errors is large and 
the sensitivityt to these errors is consequential. What 
we recommend is either changes in the payoff itself 
(clipping exposure) or the shifting of exposures away 
from that part. For that we will provide precise rules. 

The paper is organized as follows. First, we classify 
decisions according to targeted payoffs. Second, we 
discuss the problem of rare events, as these are 
the ones that are both consequential and hard to 
predict. Third, we present the classification of the 
two categories of probability distributions. Finally, we 
present the “fourth quadrant" and what we need to do 
to escape it, thus answering the call for how to handle 
“decision making under low predictability". 

3. The different types of decisions 

The first type of decisions is simple, it aims at 
“binary" payoffs, i.e. you just care whether something 
is true or false. Very true or very false does not 
matter. Someone is either pregnant or not pregnant. 
A biological experiment in the laboratory or a 
bet about the outcome of an election belong to 
this category. A scientific statement is traditionally 
considered “true" or “false” with some confidence 
interval. More technically, they depend on the zeroth 
moment, namely just on the probability of events, and 
not their magnitude — for these one just cares about 
“raw” probability. 10 


10 The difference can be best illustrated as follows. One of the 
most erroneous comparisons encountered in economics is the one 
between the “wine rating" and "credit rating" of complex securities. 
Errors in wine rating are hardly consequential for the buyer (the 
“payoff” is binary), errors in credit ratings have bankrupted banks, 
as these cany massive payoffs. 
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Clearly these are not very prevalent in life — they 
mostly exist in laboratory experiments and in research 
papers. 

The second type of decisions depends on more 
complex payoffs. The decision maker does not just 
care about the frequency, but about the impact as 
well, or, even more complex, some function of the 
impact. So there is another layer of uncertainty of 
impact. These depend on higher moments of the 
distribution. When one invests one does not care about 
the frequency, how many times he makes or loses, he 
cares about the expectation: how many times money is 
made or lost times the amount made or lost. We will 
see that there are even more complex decisions. 

More formally, where p[x] is the probability 
distribution of the random variable x, and D the 
domain on which the distribution is defined, the payoff 
A(x) is defined by integrating on D as: 

M*) = J f(x)p(x)dx. 

Note that we can incorporate utility or nonlinearities 
of the payoff in the function / (x ). But let us ignore 
utility for the sake of simplification. 

For a simple payoff, f(x) = 1. So L(x) becomes 
the simple probability of exceeding .r, since the final 
outcome is either 1 orO (or 1 and -1). 

For more complicated payoffs, f(x) can be 
complex. If the payoff depends on a simple 
expectation, i.e., A(x) = E[x], the corresponding 
function f{x) = x, and we need to ignore frequencies 
since it is the payoff that matters. One can be right 
99% of the time, but this does not matter at all, since 
with some skewed distributions, the consequences of 
the expectation of the 1% error can be too large. 
Forecasting typically has f(x) = x, a linear function 
of x, while measures such as least squares depend on 
tlie higher moments f(x) = x 2 . 

Note that some financial products can even depend 
on the fourth moment (see Table 2). n 

Next we turn to a discussion of the problem of rare 
events. 


More formally, a linear function with respect to the variable x 
has no second derivative, a convex function is one with a positive 
second derivative. By expanding the expectation of fix) we end 
up with Ef/OOl = /(x)e[Axl + l/2/"(x)£[Ax2) + . and 

hence higher orders matter to the extent of the importance of higher 
derivatives. 


4. The problem of rare events 

The passage from theory to the real world presents 
two distinct difficulties: “inverse problems” and “pre- 
asymptotics”. 

4.1. Inverse problems 

It is the greatest difficulty one can encounter in 
deriving properties. In real life we do not observe 
probability distributions, we just observe events. So 
we do not know the statistical properties — until, of 
course, after the fact — as we can sec in Fig. 1. Given 
a set of observations, plenty of statistical distributions 
can correspond to the exact same realizations — each 
would extrapolate differently outside the set of events 
on which it was derived. The inverse problem is 
more acute when more theories, more distributions 
can fit a set of data — particularly in the presence of 
nonlinearities or nonparsimonious distributions. 12 

So this inverse problem is compounded of two 
problems: 

-I- The small sample properties of rare events, as 
these will be naturally rare in a past sample. This 
is also acute in the presence of nonlincarities, 
as the families of possible models/parametrization 
explode in numbers. 

•f The survivorship bias effect of high impact rare 
events. For negatively skewed distributions (with 
a thicker left tail), the problem is worse. Clearly, 
catastrophic events will be necessarily absent from 
the data, since the survivorship of the variable 
itself will depend on such effect. Thus, left tailed 
distributions will (1) overestimate the mean; (2) 
underestimate the variance and the risk. 

Fig. 4 shows how we normally lack data in the tails; 
Fig. 5 shows the empirical effect (see Fig. 6). 

4.2. Pre-asymptotics 

Theories can be extremely dangerous when they 
were derived in idealized situations, the asymptote, but 
arc used outside the asymptote (at its limit, say infinity 


12 A Gaussian distribution is parsimonious (with only two 
parameters to lit). But the problem of adding Layers of possible 
jumps, each with a different probabilities, opens up endless 
possibilities of combinations of parameters. 
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Table 2 

Tableau of decisions. 



Mo 

Ml 

M2+ 

"True/False" 

/(*) = o 

Medicine (health not epidemics) 
Psychology experiments 

Bets (prediction markets) 
Binary/Digital derivatives 
Life/Death 

Expectations 

LINEAR PAYOFF 
/(*) = ! 

Finance: nonleveragcd investment 

Insurance, measures of expected shortfall 
General nsk management 

Climate 

Economics (Policy) 

Security: Terrorism. Natural catastrophes 
Epidemics 

Casinos 

NONLINEAR PAYOFF 
/(x) nonlinear) - x 2 ,x 3 . etc.) 

Derivative payoffs 

Dynamically hedged portfolios 

Leveraged portfolios (around the loss point) 

Cubic payoffs (strips of out of the money options) 
Errors in analyses of volatility 

Calibration of nonlinear models 

Expectation w eighted by nonlinear utility 
Kurtosis-based positioning ("volatility trading") 



Fig. 4. The confirmation bus at work. The shaded area shows what 
lend lo be missing from the observations. For negatively skewed, 
fat tailed distributions, we do not see much of negative outcomes 
for surviving entities .AND we have a small sample in the left tail. 
This illustrates why w e tend to see a better past for a certain class of 
time series than is w arranted. 
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Fig. 5. Outliers don't predict outliers. The plot shows (on a 
logarithmic scale) a shortfall in one given year against the shortfall 
the following one. repeated throughout for the 43 variables. A 
shortfall here is defined as the sum of deviations in excess of 7%. 
Past large deviations do not appear to predict future large deviations, 
at different lags. 


Log(SaW) 



Fig. 6. Regular events predict regular events. This plot shows, by 
comparison with Fig. 5. how. for the same variables, the mean 
deviation in one period predicts the one in the subsequent period. 

or the infinitesimal). Some asymptotic properties do 
work well pre-asymptotically (as we'll see, with type- 
1 distributions), which is why casinos do well, but 
others do not. particularly when it comes to the class 
of fat-tailed distributions. 

Most statistical education is based on these 
asymptotic, laboratory-style Platonic properties — yet 
we take economic decisions in the real world that very 
rarely resembles the asymptote. Most of what students 
of statistics do is assume a structure, typically with 
a known probability. Yet the problem we have is not 
so much making computations once you know the 
probabilities as finding the true distribution. 

5. The two probabilistic structures 

There are two classes of probability domains — 
very distinct qualitatively and quantitatively — ac- 
cording to precise mathematical properties. The first, 
Type-1. 
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we call ‘’benign” thin-tailed nonscalable, the second. 
Type 2, “wild” thick tailed scalable, or fractal (the at- 
tribution “wild” comes from the classification of Man- 
delbrot, 1963, 2001). 

Taleb (2009) makes the distinction along the lines 
of convergence to the Central Limit Theorem. Type-1 
converges in an acceptable form, while Type-2 either 
does not converge (infinite variance), or converges 
only in a remote asymptote and needs to be treated 
pre-asymptotically. Taleb (2009) also shows that one 
of the mistakes in the economics literature that “fattens 
the tails”, with two main classes of nonparsimonious 
models and processes (the jump-diffusion processes 
of Merton. 1976, 13 or stochastic volatility models 
such as Engels' ARCH 14 ) is to believe that the 
second type of distribution is amenable to analyses 
like the first— except with fatter tails. In reality, a 
fact commonly encountered by practitioners is that 
fat-tailed distributions are very unwieldy — as we can 
see in Fig. 2. Furthermore, we often face a problem 
of mistaking one for the other: a process that is 
extremely well behaved, but, on occasions, delivers 
a very large deviation, can easily be mistaken for a 
thin-tailed one — a problem known as the “problem 
of confirmation" (Taleb, 2007a,b). So we need to be 
suspicious of the mistake of taking Typc-2 for Type-1, 
as it is more severe (and more readily made) than the 
one in the other direction. 15 

As we saw from the data presented, this 
classification of “fat tails" does not just mean having a 
fourth moment worse than the Gaussian. The Poisson 
distribution, or a mixed distribution with a known 
Poisson jump, would have tails thicker than the 
Gaussian; but this mild form of fat tails can be dealt 
with rather easily — the distribution has all its moments 
finite. The problem comes from the structure of the 
decline in probabilities for larger deviations and the 
ease with which the tools at our disposal can be tripped 
into producing erroneous results from observations of 
data in a finite sample and jumping to wrong decisions. 


15 See the general decomposition into diffusion and jump (non- 
scalable) in Duffie. Pan. and Singleton (2000) and Merton ( 1976); 
and the discussion in Baz and Chacko (2004) and Haug (2007). 

14 Engle (1982). 

^Makridakis et al. (1993) and Makridakis and Hibon (2000) 
present evidence that more complicated methods of forecasting 
do not deliver superior results to simple ones (already bad). The 
obvious reason is that the errors in calibration swell with the 
complexity of the model. 


5.1. The scalable property of type -2 distributions 

Take a random variable x. With scalable distribu- 
tions, asymptotically, for x large enough (i.e. “in the 
tails”), depends on n, not on je (the same 

property can hold for P[X < n.v] for negative values). 
This induces statistical self-similarities. Note that ow- 
ing to the finiteness of the realizations of random vari- 
ables. and the lack of samples in the tails, we might 
not be able to observe such a property, yet not be able 
to rule out. 

For economic variables, there is no fundamental 
reason for the ratio of “exceedances" (i.e., the cumu- 
lative probability of exceeding a certain threshold) to 
decline, as both the numerator and the denominators 
are multiplied by 2. 

This self-similarity at all scales generates power- 
law, or Paretian, tails, i.e., above a crossover point, 
PIX > Jtl = Kx-".' 6 - 17 

Let us now draw the implications of type-2 
distributions. 

5.1.1. Finiteness of moments and higher order effects 

For thick tailed distributions, moments higher than 

of are not “finite”, i.e., they cannot be computed. 
They can certainly be measured in finite samples — 
thus giving the illusion of finiteness. But they typically 
show a great degree of instability. For instance, a 
distribution with an infinite variance will always 
provide, in a sample, the illusion of finitencss of 
variance. 

In other words, while errors converge for type-1 
distributions, the expectations of higher orders of x, 
say of order n, such as \fn\E{x n ], where x is the 
error, do not decline; in fact, they become explosive 
(see Fig. 7). 


16 Scalable discussions: introduced by Mandelbrot (1963). 
Mandelbrot (1997) and Mandelbrot and Taleb (in press). 

17 Complexity and power laws: Amaral ct al. (1997). Sornctte 
(2004). and Stanley. Amaral. Gopikrishnan. and Plerou (2000). 
for scalability in different aspects of financial data. Gabaix. 
Gopikrishnan. Plerou, and Stanley (2003a,b), Gabaix Ramalho. and 
Reuter (2003c). Gopikrishnan, Meyer. Amaral, and Stanley 1 1998). 
Gopikrishnan. Plerou. Amaral. Meyer, and Stanley (1999), and 
Gopikrishnan. Plerou. Gabaix. and Stanley (2000). For the statistical 
mechanics of scale free networks see Albert. Jeong. and Barabdsi 
(2000), Albert and Barabasi (2002) and BaraMsi and Albert (1999). 
The "sandpilc effect'' (i.e, avalanches and cascades) is discussed by 
Bak (1996) and Bak. Tang, and Wiesenfeld (1987. 1988), as power 
laws arise from conditions of self -organized criticality. 
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Fig. 7. Kuttosis over lime: example of an “infinite moment". The graph shows the fourth moment for crude oil in annual nonoverlapping 
observations between 1982 and 2008. The instability shows in the dependence of the measurement on the observation window. 


5.1.2. * Atypicality " of moves 

For thin tailed domains, the conditional expectation 
of a random variable X. conditional on its exceeding a 
number K. converges to K for larger values of K. 

lim E[X\ x>k \ = K. 

K —+oo 

For instance, the conditional expectation for a 
Gaussian variable (assuming a mean of 0) conditional 
on the variable exceeding 0 is approximately 0.8 
standard deviations. But with K equals 6 standard 
deviations, the conditional expectation converges to 
6 standard deviations. The same applies to all of the 
random variables that do not have a Paretian tail. This 
induces some “typicality" of large moves. 

For fat tailed variables, such a limit does not seem 
to hold: 

lim E[X\ x>k ] = ATc, 

K-+oo 

where c is a constant. For instance, the conditional 
expectation of a market move, given that it is in 
excess of 3 mean deviations, will be around 5 mean 
deviations. The expectation of a move conditional on 
it being higher than 10 mean deviations will be around 
18. This property is quite crucial. 

The atypicality of moves has the following 
significance. 

- One may correctly predict a given event, say, a 
war, a market crash, or a credit crisis. But the 
amplitude of the damage will be unpredicted. The 


open-endedness of the outcomes can cause a severe 
miscalculation of the expected payoff function. 
For instance, the investment bank Morgan Stanley 
predicted a credit crisis but was severely hurt (and 
needed to be rescued) because it did not anticipate 
the extent of the damage. 

- Methods like Value-at-Risk 18 that may correctly 
compute, say. a 99% probability of not losing 
no more than a given sum, called “value-at- 
risk”, will nevertheless miscompute the conditional 
expectation should such a threshold be exceeded. 
For instance, one has 99% probability of not 
exceeding a $1 million loss, but should such a loss 
occur, it can be $10 million or $100 million. 

This lack of typicality is of some significance. 
Stress testing and scenario generation are based on 
assuming a “crisis” scenario and checking robustness 
to it. Unfortunately such luxury is not available for fat 
tails, as “crises" do not have a typical magnitude. 

Tables 3 and 4 show the evidence of a lack 
of convergence to thin tails, and hence a lack of 
“typicality” of the moves. We stopped for segments 
for which the number of observations becomes small, 
since a lack of observations in the tails can provide the 
illusion of “thin” tails. 


18 For the definition of Value at Risk see. Jorion (2001); for a 
critique, sec Joe Noccra. “Risk Mismanagement: What led to the 
Financial Meltdown". New York Time Magarjne, Jan 2. 2009. 
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Table 3 


Conditional expectation for moves > K, 43 economic variables . 


K. Mean 
deviaUons 

Mean move (in MAD) 
in excess of K 

n 

1 

2.01443 

65.958 

2 

3.0814 

23.450 

3 

4.19842 

8,355 

4 

5.33587 

3.202 

5 

6.52524 

1.360 

6 

7.74405 

660 

7 

9.10917 

340 

8 

10.3649 

192 

9 

11.6737 

120 

10 

13.8726 

84 

11 

15.3832 

65 

12 

19.3987 

47 

13 

21.0189 

36 

14 

21.7426 

29 

15 

24.1414 

21 

16 

25.1188 

18 

17 

27.8*108 

13 

18 

31.2309 

11 

19 

35.6161 

7 

20 

35.9036 

6 


Tabled 

Conditional expectation for moves K, 43 economic variables. 


K. Mean 
deviations 

Average move (in MAD) 
below A' 

n 

-1 

-2.06689 

62.803 

-2 

-3.13423 

23.258 

-3 

—4.24303 

8.676 

-4 

-5.40792 

3.346 

-5 

-6.66288 

1.415 

-6 

-7.95766 

689 

-7 

-9.43672 

392 

-8 

-11.0048 

226 

-9 

-13.158 

133 

-10 

-14.6851 

95 

-11 

-17.02 

66 

-12 

-19.5828 

46 

-13 

-21.353 

38 

-14 

-25.0956 

27 

-15 

-25.7004 

22 

-16 

-27.5269 

20 

-17 

-33.6529 

16 

-18 

-35.0807 

14 

-19 

-35.5523 

13 

-20 

-38.7657 

11 


5.1.3. Preasymptotics 

Even if we eventually converge to a probability 
distribution of the kind well known and tractable, it is 
central that the time to convergence plays a large role. 

For instance, much of the literature invokes the 
Central Limit Theorem to assume that fat-tailed 
distributions with a finite variance converge to 
a Gaussian under summation. If daily errors are 
fat-tailed, cumulative monthly errors will become 
Gaussian. In practice, this does not appear to hold. 
The data, as we saw earlier, show that economic 
variables do not remotely converge to the Gaussian 
under aggregation. 

Furthermore, finiteness of variance is a necessary 
but highly insufficient condition. Bouchaud and 
Potters (2003) showed that the tails remain heavy 
while tlie body of the distribution becomes Gaussian 
(see Fig. 8). 

5.1.4. Metrics 

Much of time series work seems to be based on 
metrics which are in the square domain, and hence 
patently intractable. Define the norm L p : 



it will increase along with p. The numbers can become 
explosive, with rare events taking a disproportionately 
larger share of the metric at higher orders of p. 
Thus the variance/standard deviation ( p = 2), as 
a measure of dispersion, will be far more unstable 
than mean deviation (p = 1). The ratio of mean- 
deviation to variance (Taleb, 2009) is highly unstable 
for economic variables. Thus, modelizations based on 
variance become incapacitated. More practically, this 
means that for distributions with a finite mean (tail 
exponent greater than 1), the mean deviation is more 
“robust 9 


19 A Dole on ihe weaknesses of nonparametric statistics: the mean 
deviation is often used as a robust, nonparametric or distribution 
free statistic. It docs work belter than the variance, as we saw. but 
does not contain informauon on rare events, by the argument seen 
before. Likewise, nonparametric statistical methods (relying on the 
empirical frequency) will be extremely fragile to the “black swan 
problem", since the absence of large deviations in the past leave us 
in a near total opacity about their occurrence in the future— as we 
saw in Fig. 4. these arc confirmatory. In other words, nonparametric 
statistics that consist of tilting a kernel to empirical frequencies, 
assume, even more than other methods, that a large deviation will 
have a predecessor. 
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Fig. 8. Behavior of Kuitosis under aggregation: we lengthen the 
window of changes from 1 day to 50 days. Even for variables with an 
infinite fourth moment, the kuitosis tends to drop under aggregation 
in small samples, then rise abruptly after a large observation. 

5.1.5. Incidence of rare events 

One common error is to believe that thickening the 
tails leads to an increase of tlic probability of rare 
events. In fact, it usually leads to a decrease of the 
incidence of such events, but the magnitude of the 
event, should it happen, will be much larger. 

Take, for instance, a normally distributed random 
variable. The probability of exceeding 1 standard 
deviation is about 16%. Observed returns in the 
markets, with a higher kurtosis, present a lower 
probability of exceeding the same threshold, around 
7%-10%, but the depth of the excursions is greater. 

5.1.6. Calibration errors and fat tails 

One does not need to accept power laws to use 
them. A convincing argument is that if we don’t 
know what a “typical" event is, fractal power laws 
are the most effective way to discuss the extremes 
mathematically. It does not mean that the real world 
generator is actually a power law — it means that we 
don’t understand the structure of the external events 
it delivers and need a tool of analysis. Also, fractals 
simplify the mathematical discussions because all you 
need to do is to perturbate one parameter, here the a. 
and it increases or decreases the role of the rare event 
in the total properties. 

Say. for instance, that, in an analysis, you move 
of from 2.3 to 2 for data in the publishing business; 
the sales of books in excess of 1 million copies would 
triple! This method is akin to generating combinations 
of scenarios with series of probabilities and series of 
payoffs, fattening the tail at each time. 

The following argument will help illustrate the 
general problem with forecasting under fat tails. Now 


ru 40.431 



Fig. 9. Estimation error from 40 thousand economic variables. 

the problem: Parametrizing a power law lends itself 
to extremely large estimation errors (since heavy tails 
have inverse problems). Small changes in the a main 
parameter used by power laws lead to extremely large 
effects in the tails. 

And we don’t observe the a — an uncertainty that 
comes from the measurement error. Fig. 9 shows more 
than 40 thousand computations of the tail exponent a 
from different samples of different economic variables 
(data for which it is impossible to refute fractal power 
laws). We clearly have problems figuring out what 
the of is: our results arc marred by errors. The mean 
absolute error in the measurement of the tail exponent 
is in excess of 1 (i.e. between or = 2 and of = 3). 
Numerous papers in econophysics found an “average” 
alpha between 2 and 3— but if you process the >20 
million pieces of data analyzed in the literature, you 
find that the variations between single variables are 
extremely significant. 20 

Now this mean error has massive consequences. 
Fig. 10 shows the effect: the expected value of your 
losses in excess of a certain amount (called the 
“shortfall") is multiplied by >10 from a small change 
in the or that is less than its mean error. 21 


20 One aspect of this inveise problem is ev en pervasive in Monte 
Carlo experiments (much better behaved than (he real world), sec 
Weron (2001). 

21 Note that the literature on extreme value theory (Embrechts. 
Kliippelberg, & Mikosch. 1997) does not solve much of the 
problem, as the calibration enrols stay the same. The argument 
about calibration we saw earlier makes the values depend on the 
unknowable tail exponent. This calibration problem explains how 
Extreme Value Theory works better on computers than in the 
real world (and has failed completely in the economic crisis of 
2008-2009). 
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Fig. 10. The value of the expected shortfall (expected losses in 
excess of a certain threshold) in response to changes in the tail 
exponent a. We can see it explode by an order of magnitude. 

6. The map 

First quadrant: Simple binary decisions, under 
type-1 distributions: forecasting is safe. These 
situations are. unfortunately, more common in 
laboratories and games than in real life. We rarely 
observe these in payoffs in economic decision making. 
Examples: some medical decisions, casino bets, 
prediction markets. 

Second quadrant: Complex decisions under 
type-1 distributions: Statistical methods may work 
satisfactorily, though there are some risks. True, thin- 
tails may not be a panacea, owing to preasymptotics, 
lack of independence, and model error. There are 
clearly problems there, but these have been addressed 
extensively in the literature (see Freedman, 2007). 

Third quadrant: Simple decisions, under type-2 
distributions: there is little harm in being wrong — the 
tails do not impact the payoffs. 

Fourth quadrant: Complex decisions under type- 
2 distributions: this is where the problem resides. 
We need to avoid the prediction of remote payoffs — 
though not necessarily ordinary ones. Payoffs from 
remote parts of the distribution arc more difficult to 
predict than closer parts. 

A general principle is that, while in the first three 
quadrants you can use the best model you can find, this 
is dangerous in the fourth quadrant: no model should 
be better than just any model. So the idea is to exit the 
fourth quadrant. 

The recommendation is to move into the third 
quadrant — it is not possible to change the distribution: 
but it is possible to change the payoff, as will be 
discussed in Section 7 (see Table 5). 


The subtlety is that, while we have a poor idea 
about the expectation in the 4th quadrant, exposures 
to rare events arc not symmetric. 

7. Decision-making and forecasting in the fourth 
quadrant 

7.7. Solutions by changing the payoff 

Finally, the main idea proposed in this paper is 
to endogenize decisions, i.e., escape the 4th quadrant 
whenever possible by changing the payoff in reaction 
to the high degree of unpredictability and the harm it 
causes. How? 

Just consider that the property of “atypicality" 
of the moves can be compensated by truncating 
the payoffs, thus creating an organic “worst case” 
scenario that is resistant to forecast errors. Recall 
that a binary payoff is insensitive to fat tails 
precisely because above a certain level, the domain of 
integration, changes in probabilities do not impact the 
payoff. So making the payoff no longer open-ended 
mitigates the problems, thus making it more tractable 
mathematically. 

A way to express it using moments: all moments 
of the distribution become finite in the absence of 
open-ended payoffs, by putting a floor L below which 
/ (*) = 0, as well a ceiling H. Just consider that if you 
are integrating payoffs in a finite, rather than an open- 
ended domain, i.e. between L and 77, respectively, the 
tails of the distributions outside that domain no longer 
matter. Thus the domain of integration becomes the 
domain of payoff. 



With an investment portfolio, for instance, it is 
possible to “put a floor” on the payoff using insurance, 
or. even better, by changing the allocation. Insurance 
products are tailored with a maximum payoff; 
catastrophe insurance products are also set with a 
“cap”, though the cap might be high enough to allow 
for a dependence on the error of the distribution. 22 

~~ Insurance companies might cap the payoff of a single claim, but 
a collection of capped claims might represent some problems, as the 
maximum loss becomes so large as to be almost undistinguishable 
from tltat with an uncapped payoff. 
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Table 5 

The four quad rants. 

13 


Simple payoffs 

Complex payoffs 

Distribution 1 (“thin tailed") 

First quadrant: 

Second quadrant: 


Extremely safe 

Safe 

Distribution 2 (no or unknown characteristic scale) 

Thinl quadrant: 

Fourth quadrant: 


Safe 

Dangers* 


* The dangers are limited to exposures in the negative domain (i.e- adverse payoffs). Some exposures, we will see. can only be "positive''. 


7.1.1. The effect of skewness 

We omitted earlier to discuss asymmetry in either 
the payoff or the distribution. Clearly, the fourth 
quadrant can present either left or right skewness. 
If we suspect right-skewness, the true mean is more 
likely to be underestimated by the measurement of 
past realizations, and the total potential is likewise 
poorly gauged. A biotech company (usually) faces 
positive uncertainty, a bank faces almost exclusively 
negative shocks. 

More significantly, by raising the L (the lower 
bound), one can easily produce positive skewness, 
with a set floor for potential adverse outcomes and 
open upside. For instance, what Taleb (2007a) calls a 
“barbell” investment strategy consists of allocating a 
high portion of a portfolio to T-Bills (or equivalent), 
say or, with 0 < a < 1. and a small portion (1 — or) to 
high-variance securities. While the total portfolio has 
medium variance, L - (1 - or) times the face value 
invested, another portfolio of the same variance might 
lose 100%. 

7.1.2 . Convex and concave to error 

If a source of uncertainty can offer more benefits 
than a potential harm, then there may be gains from 
it — which we label “convex” or “concave”. 

More generally, we can be concave to model error 
if tlie payoff from the error (obtained by changing 
the tails of the distribution) has a negative second 
derivative with respect to the change in the tails, or is 
negatively skewed (like the payoff of a short option). It 
will be convex if the payoff is positively skewed (like 
the payoff of a long option). 

7.1.3. The effect of leverage in operations and 
investment 

Leveraging in finance has the effect of increasing 
concavity to model error. As we will see, it is exactly 
the opposite of redundancy — it causes payoffs to 


increase, but at the costs of an absorbing barrier should 
there be an extreme event that exceeds file allowance 
made in the risk measurement. Redundancy, on the 
other hand, is the equivalent of de-lever aging. i.e. by 
having more idle ‘inefficient" capital on the side. But 
a a second look at such funds can reveal that there may 
be a direct expected value from being able to benefit 
from opportunities in the event of asset deflation, and 
hence “idle" capital needs to be analyzed as an option. 

7.2. Solutions by mitigating forecasting errors 

7.2.1. Optimization vs. redundancy 

The optimization paradigm of the economics 
literature meets some problems in the fourth quadrant: 
what if we haw a consequential forecasting error? 
Aside from the issue that the economic agent is 
optimizing on the future states of the world, with 
a given probability distribution, nowhere 2 -’ have the 
equations taken into account the possibility of a 
large deviation that would allow not optimizing 
consumption and having idle capital. Also, the 
psychological literature on well-being (Kahneman, 
1999) shows an extremely concave utility function of 
income — if one spends such income. But if one hides 
it under the mattress, one will be less vulnerable to 
an extreme event. So there is an enhanced survival 
probability for those who have additional margin. 

While economics have been mired in conventional 
linear analysis, stochastic optimization with Bellman- 
style equations that fall into the category Type-1, a 
different point of view is provided by complex systems 
analysis. One of the central attributes of complex 
systems is redundancy (May, Levin. <& Sugihara, 
2008). 


See Mellon (.1992) for a discussion of the general consumption 
Capital Asset Pacing Market. 
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Biological systems — those that have survived 
millions of years — include a large share of 
redundancies. 2 ' 1 ■ 25 Just consider the number of double 
organs (lungs, kidneys, ears). This may suggest an 
option-theoretic analysis: redundancy is like an option. 
One certainly pays for it, but it may be necessary 
for survival. And while redundancy means similar 
functions used by identical organs or resources, 
biological systems have, in addition, recourse to 
“degeneracy”, the possibility of one organ to perform 
more than one function, which is the analog of 
redundancy at a functional level (Edelman & Gaily, 
2001). 

When institutions such as banks optimize, they 
often do not realize that a simple model error can blow 
through their capital (as it just did) (see Fig. 1 1). 

Examples: In one day in August 2007, Goldman 
Sachs experienced 24 times the average daily 
transaction volume 26 — would 29 times have blown 
up the clearing system? Another severe instance of 
an extreme “spike” lies in an event of September 
18, 2008. in the aftermath of the Lehman Bothers 
Bankruptcy. According to congress documents, only 
made public in February 2009: 

On Thursday (Sept 18), at 11 am the Federal Reserve 
noticed a tremendous draw-down of money market 
accounts in the US, to the tune of $550 billion 27 was 
being drawn out in the matter of an hour or two. 

If they had not done that [add liquidity], their 
estimation is that by 2 pm that afternoon, $5.5 trillion 
would have been drawn out of the money market system 
of the U.S., which would have collapsed the entire 
economy of the U.S., and within 24 li the wodd economy 
would have collapsed. It would have been the end of our 
economic system and our political system as we know 
it. 28 

For naive economics, the best way to effectively 
reduce costs is to minimize redundancy, and hence 
avoiding the option premium of insurance. Indeed, 

24 MayetaL(2008). 

25 For the scalability of biological systems, see Burlando (1993), 
Enquist and Niklas (2001), Harte. Kinzig. and Green (1999). Ritchie 
and Olff (1999) and Soil. Manrubia. Benton. Kauffman, and Bak 
. 19990 , 

26 Personal communication. Pentagon Highland Forum. April 
meeting. 2008. 

27 Even if the number, as is possible, is off by one order of 
magnitude, the consequences remain extremely severe. 

2 8 http://www.liveleakjCom/vicw?i=ca2 .12 3403228 1 . 


Typ«-i Noise 



Fig. 11. Comparison between Gaussian style noise and iype-2 
noise with extreme spikes — which necessitates more redundancy 
(or insurance) than normally required. Policymakers and forecasters 
were not aware that complex systems tend to produce the second 
type of noise. 

some systems tend to optimize and therefore become 
more fragile. Albert and Barabasi (2002) and BarabSsi 
and Albert (1999) warned (ahead of the North Eastern 
power outage of August 2003) how electricity grids, 
for example, optimize to the point of not coping with 
unexpected surges — which predicted the possibility of 
a blackout of the magnitude of the one that took place 
in the North Eastern U.S. in August 2003. We cannot 
discuss “flat earth” globalization without realizing 
that it is overoptimized to the point of maximal 
vulnerability. 

7.2.2. Time and sample size 

It takes much, much longer for a fat-tailed time 
series to reveal its properties — in fact, many can, 
in short episodes, masquerade as thin-tailed. At the 
worst, we don’t know how long it would take to 
know. But we can have a pretty clear idea whether 
organically , because of the nature of the payoff, the 
“Black Swan" can hit on the left (losses) or on the 


Please cite this article in press as: Taleb, N. N. Errors, robustness, and the fourth quadrant. International Journal of Forecasting (2009). 
doi: 10.101 6/jj)forecast.2009.05.027 






54 


ARTICLE IN PRESS 


N.N. Taleb /International Journal of Forecasting I Jill. Ill III IS 


right (profits). This point can be used in climatic 
analysis. Things that have worked for a long time are 
preferable — they are more likely to have readied their 
ergodic states. 

Likewise, portfolio diversification needs to be 
larger, much larger than anticipated. A mean variance 
Markowitz-style portfolio construction fails in the 
real world on several accounts. Taleb (2009) shows 
that, even if we assume finite variance, but fat tails 
and an unknown variance, the process of discovery 
of the variance itself makes portfolio theory totally 
unusable. DeMiguel, Garlappi, and Uppal (2007) 
show that a naive 1 fn allocation outperforms out-of- 
sample any form of “optimal’’ portfolio — compatible 
with the notion that fat tails (and unknown future 
properties from past samples) require much broader 
diversification than is required by modem portfolio 
theory. 

7.2.3. The problem of moral hazard 

It is optimal (both economically and psychologi- 
cally) to make a series of annual bonuses betting on 
hidden risks in the fourth quadrant, then “blow up” 
(Taleb, 2004). The problem is that bonus payments are 
made with a higher frequency (i.e. annually) than is 
warranted from the statistical properties (when it takes 
longer to capture the statistical properties). 

7.2.4. Metrics 

Conventional metrics based on type 1 randomness 
fail to produce reliable results — while the economics 
literature is grounded in tliem. Concepts like “standard 
deviation" are not stable and do not measure anything 
in the fourth quadrant. This is also true for “linear 
regression" (the errors are in the fourth quadrant), 
“Sharpe ratio", the Markowitz optimal portfolio, 29 
ANOVA. Least squares, etc. “Variance" and “standard 
deviation" are terms invented years ago when we had 
no computers. Note that from the data shown and the 
instability of the kurtosis, no sample will ever deliver 
the true variance in a reasonable time. However, 
note that truncating payoffs blunts the effects of the 
inadequacy of the metrics. 


“ 9 The framework of Markowitz (1952), as it is built on the l? 
norm, does not stand any form of empirical or even theoretical 
validity, owing to the dominance of higher moment effects, even 
in the presence of “finite" variance, see Taleb (2009). 


8. Conclusion 

To conclude, wc offered a method of robust ifying 
payoffs from large deviations and making forecasts 
possible to perform. The extensions can be generalized 
to a larger notion of society’s safety — for instance how 
we should build systems (internet, banking structure, 
etc.) to be impervious to random effects. 
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Chairman Miller. Thank you, Dr. Taleb. 

Dr. Bookstaber for five minutes. 

STATEMENT OF DR. RICHARD BOOKSTABER, FINANCIAL 

AUTHOR 

Dr. Bookstaber. Mr. Chairman and Members of the Committee, 
I thank you for the opportunity to testify today. My oral testimony 
will begin with a discussion of the limitations of VaR. I will then 
discuss the role of VaR in the recent market meltdown and con- 
clude with suggestions for filling the gap left by the limitations of 
VaR. 

The limitations of VaR are readily apparent by looking at the 
critical assumptions behind it. For the standard construction of 
VaR, these assumptions are, first, that all portfolio positions are in- 
cluded; secondly, that the sample history used in VaR is a reason- 
able representation of things that are likely to occur going forward; 
and third, that the normal distribution function that it uses is a 
reasonable representation of the statistical distribution underlying 
the returns. These assumptions are often violated, leading VaR es- 
timates to be misleading. So let me discuss each of these in turn. 

First of all, in terms of incomplete positions, obviously, for risk 
to be measured, all the risky positions must be included in the 
analysis, but for larger institutions, it is commonplace for some po- 
sitions to be excluded. This can happen because the positions are 
held off a balance sheet beyond the purview of those doing the risk 
analysis, because they are in complex instruments that have not 
been sufficiently modeled, or because they are in new so-called in- 
novative products that have yet to be added into the risk process. 
This provides a compelling reason to have what I call ‘flight to sim- 
plicity’ in financial products, to move away from complex and cus- 
tomized innovative products and towards standardization. 

In terms of unrepresentative sample periods, VaR gives a meas- 
ure of risk that assumes tomorrow is drawn from the same dis- 
tribution as the sample data used to compute the VaR. If the fu- 
ture does not look like the past — in particular, if a crisis emerges, 
VaR will no longer be a good measure of risk, which is to say that 
VaR is a good measure of risk except when it really matters. 
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Third, in terms of fat tails and normal distribution, largely be- 
cause of crisis events, security returns tend to have fatter tails 
than what is represented by a normal distribution. That is, there 
tend to be more outliers and extreme events than a normal dis- 
tribution would imply. Now, one way to address this well-known in- 
accuracy is to modify the distribution allowing for fatter tails, but 
this adds complication to VaR analysis while contributing little in- 
sight in terms of risk. 

A better approach is to accept the limitations of VaR, and then 
try to understand the market crises where VaR fails. If we under- 
stand the dynamics of market crises, we may be able to improve 
risk management to make it work when it is of the greatest impor- 
tance. A starting point for understanding financial market crises is 
leverage and the crowding of trades. These lead to the common cri- 
sis dynamic — what I call a liquidity crisis cycle. Such a cycle begins 
when there is some exogenous shock that causes a drop in a mar- 
ket that is crowded with leveraged investors. The highly leveraged 
investors are forced to sell to meet their margin requirements. 
Their selling drops prices further, which in turn forces yet more 
selling, resulting in a cascading cycle downward in prices. Now, the 
investors that are under pressure discover there is no longer any 
liquidity in the stressed market, so they start to liquidate their po- 
sitions in other markets to generate the required margin. And if 
many investors that are in the first market also have high expo- 
sure in a second one, the downward spiral propagates to this sec- 
ond market. 

This phenomenon explains why a crisis can spread in surprising 
and unpredictable ways. The contagion is primarily driven by what 
other securities are owned by the funds that need to sell. For ex- 
ample, a simple example of this is what happened with the silver 
bubble back in 1980. The silver market became closely linked with 
the market for cattle. Why? Because the Hunt family had margin 
calls on their silver position, and so they sold whatever else they 
could, and what else they had to sell happened to be cattle. So thus 
there was a contagion based not on any economic linkage but based 
on who was under pressure and what else they owned. 

Now, this cycle evolves unrelated to historical relationships, out 
of the reach of VaR-type models. But that doesn’t mean it is beyond 
analysis. But if we want to analyze it, we need to know the lever- 
age and the positions of the major market participants. Gathering 
these critical data is the first step in measuring and managing cri- 
sis risk, and should be the role of a market regulator. 

Now, let me talk specifically about the role of VaR in the current 
crisis. Whatever the limitations of VaR models, they were not the 
key culprits in the case of the multi-billion dollar write-downs cen- 
tral to the current crisis. The large bank inventories were there to 
be seen. You didn’t need to have any models or sophisticated detec- 
tive or forensic work to see them. Furthermore, it was clear that 
these inventories were illiquid and that their market values were 
uncertain. It is hard to understand how this elephant in the room 
was missed, how a risk manager could see inventory grow from a 
few billion dollars to 10 billion dollars and then to 30 or 40 billion 
dollars, and not take action to bring that inventory down. 
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One has to look beyond VaR to sheer stupidity or collective man- 
agement failure. The risk managers missed the growing inventory, 
or did not have the courage of their conviction to insist on its being 
reduced, or the senior management was not willing to heed their 
demands. Whatever the reason, VaR was not central to the crisis. 
Focus would be better placed on failures in risk governance than 
failures of risk models, whatever the flaws of VaR are. 

Now, in summary, let me first emphasize, I believe that VaR 
does have value. If one were forced to pick a single number for the 
risk of a portfolio in the near future, VaR would be a good choice 
for the job. VaR illuminates most of the risk landscape, but, unfor- 
tunately, the places its light fails to reach are the canyons, crevices 
and cliffs. 

So we can do two things to try to improve on and address the 
limitations of VaR. One is to employ coarser measures of risk, 
measures that have fewer assumptions and that are less dependent 
on the future looking like the past. The use of the leverage ratio 
mandated by U.S. regulators is an example of such a measure. The 
leverage ratio does not overlay assumptions about the correlation 
or the volatility of the assets, and does not assume any mitigating 
effects from diversification. It does, however, have its own limita- 
tions as a basis for capital adequacy. The second is to add other 
risk methods that are better at illuminating the areas VaR does 
not reach. So in addition to measuring risk using a standard VaR 
approach, develop scenarios for crises and test capital adequacy 
under those scenarios. Critical, of course, to the success of this ap- 
proach is the ability to ferret out potential crises and describe them 
adequately for risk purposes. We can go a long way toward this 
goal by having regulators amass and aggregate data on the posi- 
tions and leverage of large financial institutions. These data are 
critical because we cannot manage what we cannot measure, and 
we cannot measure what we cannot see. With these data, we will 
be better able to measure the crowding and leverage that leads to 
crisis, and shed light on risks that fail to be illuminated by VaR. 

Let me close my oral comments by responding to comments by 
both the Chairman and the Ranking Member. The analogy of VaR 
and the models related to risk to models used in other engineering 
and physical systems — I think there is a critical distinction be- 
tween financial systems and other engineering systems, because fi- 
nancial systems are open to gaming. If I discover a valve that is 
poorly designed in a nuclear power plant and design a new valve 
to replace it, and install that valve, the valve doesn’t sit there and 
try to figure out if it can fool me into thinking it is on when it is 
really off. But in the financial markets, that is what happens. So 
any engineering solution or any analogy to physical processes is 
going to be flawed when they are applied to the financial markets, 
because those in the financial markets can game against the sys- 
tem to try to find ways around any regulation, and to find other 
ways to do what they want to do. And I believe that one of the key 
tools for this type of gaming are sophisticated, innovative, complex 
products that can often obfuscate what people are doing. 

So, I think, parenthetical to the issues of VaR and other models 
is, number one, the recognition that no model can work completely 
in the financial markets the way they can in other physical sys- 
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terns, and number two, that if we want to curb or diminish the 
issues of gaming, we have to have more simplicity and trans- 
parency in the financial instruments. 

Thank you. I look forward to your questions. 

[The prepared statement of Dr. Bookstaber follows:] 

Prepared Statement of Richard Bookstaber 

Mr. Chairman and Members of the Committee, I thank you for the opportunity 
to testify today. My name is Richard Bookstaber. Over the past decade I have 
worked as the risk manager in two of the world’s largest hedge funds, Moore Capital 
Management and, most recently, Bridgewater Associates. In the 1990s I oversaw 
firm-wide risk at Salomon Brothers, which at the time was the largest risk-taking 
firm in the world, and before that was in charge of market risk at Morgan Stanley. 

I am the author of A Demon of Our Own Design — Markets, Hedge Funds, and the 
Perils of Financial Innovation. Published in April, 2007, this book warned of the po- 
tential for financial crisis resulting from the growth of leverage and the proliferation 
of derivatives and other innovative products. 

Although I have extensive experience on both the buy-side and sell-side, I left my 
position at Bridgewater Associates at the end of 2008, and come before the Com- 
mittee in an unaffiliated capacity, representing no industry interests. 

My testimony will discuss what VaR is, how it can be used and more importantly, 
how it can be misused. I will focus on the limitations of VaR in measuring crisis 
risk. I will then discuss the role of VaR in the recent market meltdown, concluding 
with suggestions for ways to fill the gaps left by the limitations of VaR. 

What is VaR? 

VaR, or Value-at-Risk, measures the risk of a portfolio of assets by estimating the 
probability that a given loss might occur. For example, the dollar VaR for a par- 
ticular portfolio might be expressed as “there is a ten percent probability that this 
portfolio will lose more than $VaR over the next day.” 

Here is a simplified version of the steps in constructing a VaR estimate for the 
potential loss at the ten percent level: 

1. Identify all of the positions held by the portfolio. 

2. Get the daily returns for each of these positions for the past 250 trading days 
(about a one-year period). 

3. Use those returns to construct the return to the overall portfolio for each day 
over the last 250 trading days. 

4. Order the returns for those days from the highest to the lowest, and pick 
the return for the day that is the 25th worst day’s return. That will be a 
raw estimate of the daily VaR at the ten percent level. 

5. Smooth the results by fitting this set of returns to the Normal distribution 
function. 1 

Limitations of VaR 

The critical assumptions behind the construction of VaR are made clear by the 
process described above: 

1. All of the portfolio positions are included. 

2. The sample history is a reasonable representation of what things will look 
like going forward. 

3. The Normal distribution function is a reasonable representation of the statis- 
tical distribution underlying the returns. 

The limitations to VaR boil down to issues with these three assumptions, assump- 
tions that are often violated, leading VaR estimates to be misleading. 


1 The risk for a Normal distribution is fully defined by the standard deviation, and the results 
from Step 3 can be used to estimate the standard deviation of the sample. If the estimated 
standard deviation is, say, five percent, then the VaR at the ten percent level will be a loss of 
eight percent. For a Normal distribution the ten percent level is approximately 1.6 standard de- 
viations. 
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Incomplete positions 

Obviously, risk cannot be fully represented if not all of the risky positions are in- 
cluded in the analysis. But for larger institutions, it is commonplace for this to 
occur. Positions might be excluded because they are held off-balance sheet, beyond 
the purview of those doing the risk analysis; they might be in complex instruments 
that have not been sufficiently modeled or that are difficult to include in the posi- 
tion database; or they might be in new products that have not yet been included 
in the risk process. In the recent crisis, some banks failed to include positions in 
collateralized debt obligations (CDOs) for all three of these reasons . 2 And that exclu- 
sion was not considered an immediate concern because they were believed to be low 
risk, having attained a AAA rating. 

The inability to include all of the positions in the VaR risk analysis, the most ru- 
dimentary step for VaR to be useful, is pervasive among the larger institutions in 
the industry. This provides a compelling reason to have a ‘flight to simplicity’ in fi- 
nancial products, to move away from complex and customized innovative products 
and toward standardization . 3 

Unrepresentative sample period 

VaR gives a measure of risk that assumes tomorrow is drawn from the same dis- 
tribution as the sample data used to compute the VaR. If the future does not look 
like the past, in particular if a crisis emerges, then VaR will no longer be a good 
measure of risk . 4 Which is to say that VaR is a good measure of risk except when 
it really matters . 5 

It is well known that VaR cannot measure crisis risk. During periods of crisis the 
relationship between securities changes in strange and seemingly unpredictable 
ways. VaR, which depends critically on a set structure for volatility and correlation, 
cannot provide useful information in this situation. It contains no mechanism for 
predicting the type of crisis that might occur, and does not consider the dynamics 
of market crises. This is not to say that VaR has no value or is hopelessly flawed. 
Most of the time it will provide a reasonable measure of risk — indeed the vast ma- 
jority of the time this will be the case. If one were forced to pick a single number 
for the risk of a portfolio in the near future, VaR would be a good choice for the 
job. VaR illuminates most of the risk landscape. But unfortunately, the places its 
light fails to reach are the canyons, crevices and cliffs. 

Fat Tails and the Normal Distribution 

Largely because of crisis events, security returns tend to have fatter tails than 
what is represented by a Normal distribution. That is, there tend to be more 
outliers and extreme events than what a Normal distribution would predict. This 
leads to justifiable criticism of VaR for its use of the Normal distribution. However, 
sometimes this criticism is overzealous, suggesting that the professionals who as- 
sume a Normal distribution in their analysis are poorly trained or worse. Such criti- 
cism is unwarranted; the limitations of the Normal distribution are well-known. I 
do not know of anyone working in financial risk management, or indeed in quan- 
titative finance generally, who does not recognize that security returns may have 
fat tails. It is even discussed in many investment textbooks, so it is a point that 
is hard to miss . 6 


2 Regulatory capital on the trading assets that a bank does not include in VaR — or for which 
the bank’s VaR model does not pass regulatory scrutiny — is computed using a risk-rating based 
approach. However, the rating process itself suffers from many of the difficulties associated with 
calculating VaR, as illustrated by the AAA ratings assigned to many mortgage-backed CDOs 
and the consequent severe underestimation of the capital required to support those assets. 

3 1 discuss the complexity and related risk issues surrounding derivatives and related innova- 
tive products in Testimony of Richard Bookstaber, Submitted to the Senate of the United States, 
Committee on Agriculture, Nutrition, and Forestry for the Hearing: “Regulatory Reform and the 
Derivatives Markets,” June 4, 2009. 

4 One way to try to overcome the problem of relying on the past is to use a very long time 
period in the VaR calculation, with the idea that a longer period will include many different 
regimes, crises and relationships. Such a view misses the way different regimes, essentially dif- 
ferent distributions, mix to lead to a final result. A long time period gives muddied results. To 
see this, imagine the case where in half of the past two assets were strongly positively cor- 
related and the other half they were strongly negatively correlated. The mixing of the two would 
suggest the average of little correlation, thus giving a risk posture that did not exist in either 
period, but that also incorrectly suggests diversification opportunities. 

5 As a corollary to this, one could also say that diversification works except when it really mat- 
ters. 

6 For example. Investments, by Bodie, Kane, and Marcus, 8th edition (McGraw-Hill/Irwin), has 
a section (page 148) entitled “Measurement of Risk with Non-normal Distributions.” 
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The issue is how this well-known inaccuracy of the Normal distribution is ad- 
dressed. One way is knowingly to misuse VaR, to ignore the problem and act as if 
VaR can do what it cannot. Another is to modify the distribution to allow for fatter 
tails. 7 This adds complication and obfuscation to the VaR analysis, because any ap- 
proach employing a fat-tailed distribution increases the number of parameters to es- 
timate, and this increases the chance that the distribution will be mis-specified. And 
in any case, simply fattening up the tails of the distribution provides little insight 
for risk management. 

I remember a cartoon that showed a man sitting behind a desk with a name plate 
that read ‘Risk Manager.’ The man sitting in front of the desk said, “Be careful? 
That’s all you can tell me, is to be careful?” Stopping with the observation that ex- 
treme events can occur in the markets and redrawing the distribution accordingly 
is about as useful as saying “be careful.” A better approach is to accept the limita- 
tions of VaR, and then try to understand the nature of the extreme events, the mar- 
ket crises where VaR fails. If we understand the dynamics of market crisis, we may 
be able to improve risk management to make it work when it is of the greatest im- 
portance. 

Understanding the Dynamics of Market Crises 

A starting point for understanding financial market crises is leverage and the 
crowding of trades, both of which have effects that lead to a common crisis dynamic, 
the liquidity crisis cycle. 

Such a cycle begins when an exogenous shock causes a drop in a market that is 
crowded with leveraged investors. The highly leveraged investors are forced to sell 
to meet their margin requirements. Their selling drops prices further, which in turn 
forces yet more selling, resulting in a cascading cycle downward in prices. Those in- 
vestors that are under pressure discover there is no longer liquidity in the stressed 
market, so they start to liquidate their positions in other markets to generate the 
required margin. If many of the investors that are in the first market also have high 
exposure in a second one, the downward spiral propagates to this second market. 8 

This phenomenon explains why a crisis can spread in surprising and unpredict- 
able ways. The contagion is driven primarily by what other securities are owned by 
the funds that need to sell. 9 For example, when the silver bubble burst in 1980, the 
silver market became closely linked to the market for cattle. Why? Because when 
the Hunt family had to meet margin calls on their silver positions, they sold what- 
ever else they could. And they happened also to be invested in cattle. Thus there 
is contagion based not on economic linkages, but based on who is under pressure 
and what else they are holding. 

This cycle evolves unrelated to historical relationships, out of the reach of VaR- 
type models. But that does not mean it is beyond analysis. Granted it is not easy 
to trace the risk of these potential liquidity crisis cycles. To do so with accuracy, 
we need to know the leverage and positions of the major market participants. No 
one firm, knowing only its own positions, can have an accurate assessment of the 
crisis risk. Indeed, each firm might be managing its risk prudently given the infor- 
mation it has at its disposal, and not only miss the risk that comes from crowding 
and leverage, but also unwittingly contribute to this risk. Gathering these critical 
data is the first step in measuring and managing crisis risk. This should be the role 
of a market regulator. 10 


7 Extreme value theory is the bastion for techniques that employ distributions with a higher 
probability of extreme events. 

8 The use of VaR -based capital can actually contribute to this sort of cycle. VaR will increase 
because of the higher volatility — and also possibly because of the higher correlations — leading 
potential liquidity providers and lenders to pull back. This was a likely exacerbating effect dur- 
ing the 1997 Asian crisis. 

9 As an illustration, the proximate cause of Long Term Capital Management’s (LTCM’s) de- 
mise was the Russian default in August, 1998. But LTCM was not highly exposed to Russia. 
A reasonable risk manager, aware of the Russian risks, might not have viewed it as critical to 
LTCM. But the Russian default hurt LTCM because many of those who did have high leverage 
in Russia also had positions in other markets where LTCM was leveraged. When the Russian 
debt markets failed and these investors had to come up with capital, they sold their more liquid 
positions in, among other things, Danish mortgage bonds. So the Danish mortgage bond market 
and these other markets went into a tail spin, and because LTCM was heavily exposed in these 
markets, the contagion took LTCM with it. 

10 1 discuss the need for firm-level position and leverage data in crisis risk management in 
previous testimony before both the House and the Senate. For example, Testimony of Richard 
Bookstaber, Submitted to the Congress of the United States, House Financial Services Com- 
mittee, for the Hearing: “Systemic Risk: Examining Regulators Ability to Respond to Threats 

Continued 
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The Role of VaR in the Current Crisis 

The above discussion provides part of the answer to the question of the role of 
VaR in the current market crisis: If VaR was used as the source of risk measure- 
ment, and thus as the determinant of risk capital, then it missed the potential for 
the current crisis for the simple reason that VaR is not constructed to deal with cri- 
sis risk. And if VaR was applied as if it actually reflected the potential for crisis, 
that is, if it was forgotten that VaR is only useful insofar as the future is drawn 
from the same distribution as the past, then this led to the mis-measurement of 
risk. So if VaR was the sole means of determining risk levels and risk capital com- 
ing into this crisis, it was misused. But this does not present the full story. 

Whatever the limitations of VaR models, they were not the key culprits in the 
case of the multi-billion dollar write-downs during the crisis. The large bank inven- 
tories were there to be seen; no models or detective work were needed. Furthermore, 
it was clear the inventories were illiquid and their market values uncertain . 11 It is 
hard to understand how this elephant in the room was missed, how a risk manager 
could see inventory grow from a few billion dollars to ten billion dollars and then 
to thirty or forty billion dollars and not react by forcing that inventory to be brought 
down. 

Of course, if these inventories were not properly included in the VaR analysis, the 
risk embodied by these positions would have been missed, but one has to look be- 
yond VaR, to culprits such as sheer stupidity or collective management failure: The 
risk managers missed the growing inventory, or did not have the courage of their 
conviction to insist on its reduction, or the senior management was not willing to 
heed their demands. Whichever the reason, VaR was not central to this crisis . 12 
Focus would be better placed on failures in risk governance than failures of risk 
models. 

Summary: VaR and Crisis Risk 

There are two approaches for moving away from over-reliance on VaR. 

The first approach is to employ coarser measures of risk, measures that have 
fewer assumptions and that are less dependent on the future looking like the past . 13 
The use of the Leverage Ratio mandated by U.S. regulators and championed by the 
FDIC is an example of such a measure . 14 The leverage ratio does not overlay as- 
sumptions about the correlation or the volatility of the assets, and does not assume 
any mitigating effect from diversification, although it has its own limitations as a 
basis for capital adequacy . 15 

The second approach is to recognize that while VaR provides a guide to risk in 
some situations, it must be enhanced with other measures that are better at illu- 
minating the areas it does not reach. For example, Pillar II of Basel II has moved 
to include stress cases for crises and defaults into its risk capital process. So in addi- 
tion to measuring risk using a standard VaR approach, firms must develop sce- 


to the Financial System,” October 2, 2007, and Testimony of Richard Bookstaber, Submitted to 
the Senate of the United States, Senate Banking, Housing and Urban Affairs Subcommittee on 
Securities, Insurance and Investment, for the Hearing: “Risk Management and Its Implications 
for Systematic Risk,” June 19, 2008. 

11 This is especially true when one considers the business of the banks, which is to package 
the securities and sell them. The growth of inventory was outside the normal business of the 
banks. That the securities were not moving out the door should have been an immediate indica- 
tion they were not correctly priced. 

12 Indeed, in some important cases, VaR was not even employed in the risk process. A case 
in point is the ‘super senior’ mortgage CDO positions which caused huge trading losses at a 
number of banks. There is a common misconception that regulatory capital for trading assets 
is automatically computed using VaR. In fact, trading assets are eligible for VaR-based capital 
only if the bank can demonstrate to its supervisor that its model is robust. Absent this, a coars- 
er method is applied. Many of the highly complex securities at the heart of the recent crisis 
were not regarded as being suitable for VaR treatment, and received a simpler ratings-based 
treatment, which proved to severely underestimate the capital required to support the assets. 

13 1 believe coarse measures — measures that are not fine tuned to be ideal in any one environ- 
ment, but are robust across many environments — are a key to good risk management. 

14 The Leverage Ratio is the ratio of Tier 1 capital, principally equity and retained earnings, 
to total assets. 

15 The Leverage Ratio is inconsistent with Basel II because it is not sensitive to the riskiness 
of balance sheet assets and it does not capture off-balance sheet risks. By not taking the relative 
risk of assets into account, it could lead to incentives for banks to hold riskier assets, while on 
a relative basis penalizing those banks that elect to hold a low-risk balance sheet. In terms of 
risk to a financial institution, the time horizon of leverage is also important, which the Leverage 
Ratio also misses. The problems with Bear Stearns and Lehman was not only one of leverage 
per se, but of funding a sizable portion of leverage in the short-term repo market. They thus 
were vulnerable to funding drying up in the face of a crisis. 
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narios for crises and test their capital adequacy under those scenarios. Critical to 
the success of this approach is the ability to ferret out potential crises and describe 
them adequately for risk purposes. 

This means that for crisis-related stress testing to be feasible, we first must be- 
lieve that it is indeed possible to model financial crisis scenarios, i.e., that crises are 
not ‘black swans.’ This is not to say that surprises do not occur. Though recently 
popularized, the recognition that we are beset by unanticipatable risk, by events 
that seemingly come out of nowhere and catch us unawares, has a long history in 
economics and finance, dating back to Frank Knight in the 1920s. 16 The best de- 
fense against such risks is to maintain a coarse, simple and robust financial struc- 
ture. Rather than fine-tuning for the current environments, we need risk measures 
and financial instruments which, while perhaps not optimal for the world of today, 
will be able to operate reasonably if the world changes in unexpected ways. VaR 
as currently structured is not such a risk measure. 

However, although surprises do occur, crisis scenarios are not wholly 
unanticipatable; they are not in the realm of Knightian uncertainty. We have had 
ample experience with financial crises. We know a thing or two about them. 17 And 
we can further anticipate crisis risk by amassing data on the positions and leverage 
of the large investment firms. The regulator is best suited to take on this task, be- 
cause these are data that no one firm can or should fully see. 18 With these critical 
data we will be better able to measure the crowding and leverage that lead to li- 
quidity crisis cycles and begin to shed light on the areas of financial risk that fail 
to be illuminated by VaR. 19 


16 Knight makes the distinction between risks we can identify and measure and those that 
are unanticipatable and therefore not measurable in Risk, Uncertainty, and Profit. (1921), Bos- 
ton, MA: Houghton Mifflin Company. 

17 For example, even beyond the insights to be gained from a detailed knowledge of firm-by- 
firm leverage and market crowding, there are some characteristics of market crisis that can be 
placed into a general scenario. When a crisis occurs, equity prices drop, credit spreads rise, and 
the volatility of asset returns increases. The yield curve flattens and gold prices rise. Further- 
more, the correlation between individual equities rises, as does the correlation between equities 
and corporate bonds. The riskier and less liquid assets fare more poorly, so, for example, emerg- 
ing markets take a differentially bigger hit than their G-7 cousins. More broadly, anything that 
is risky or less liquid becomes more common and negative in its return; the subtleties of pricing 
between assets becomes overshadowed by the assets’ riskiness. However, short-term interest 
rates and commodity prices are less predictable; in some cases, such as in the case of the infla- 
tion-laden crisis of 1973—1974, they rise, while in other cases, such as in the current crisis, they 
drop. 

Each of these effects can occur with a ferocity far beyond what is seen in normal times, so if 
these crisis events are overlaid on the distribution coming out of the VaR model based on those 
normal times one will come away saying the crisis is a 100-year flood event, a twenty standard 
deviation event, a black swan. But it is none of these things. It is a financial crisis, and such 
crises occur frequently enough that to be understood without such shock and awe. 

18 Financial firms will be justifiably reticent to have their position and leverage information 
made public, so the collection and analysis of the data will have to reside securely in the regu- 
lator. 

19 With these data, the regulator is also in a position to run risk analysis independent of the 
firms. Under Basel II, the regulator still depends on the internal processes of the banks for the 
measurement of risk and the resulting capital requirements. 
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Appendix 

Related Blog Posts on VaR and Risk Management 

The Fat-Tailed Straw Man 

See http: 1 1 rick, bookstaber. com / 2009 103/ fat-tailed-straw-man. html 

My Time article about the quant meltdown of August, 2007 started with “Looks 
like Wall Street’s mad scientists have blown up the lab again.” Articles on Wall 
Street’s mad scientist blowing up the lab seem to come out every month in one 
major publication or another. The New York Times has a story along these lines 
today and had a similar story in January. 

There is a constant theme in these articles, invariably including a quote from 
Nassim Taleb, that quants generally, and quantitative risk managers specifically, 
missed the boat by thinking, despite all evidence to the contrary, that security re- 
turns can be modeled by a Normal distribution. 

This is a straw man argument. It is an attack on something that no one believes. 

Is there anyone well trained in quantitative methods working on Wall Street who 
does not know that security returns have fat tails? It is discussed in most every in- 
vestment text book. Fat tails are apparent — even if we ignore periods of crisis — in 
daily return series. And historically, every year there is some market or other that 
has suffered a ten standard deviation move of the “where did that come from” vari- 
ety. I am firmly in the camp of those who understand there are unanticipatable 
risks; as far back as an article I co-authored in 1985, I have argued for the need 
to recognize that we face uncertainty from the unforeseeable. To get an idea of how 
far back the appreciation of this sort of risk goes in economic thought, consider the 
fact that it is sometimes referred to as Knightian uncertainty. 

Is there any risk manager who does not understand that VaR will not capture 
the risk of market crises and regime changes? The conventional VaR methods are 
based on historical data, and so will only be an accurate view of risk if tomorrow 
is drawn from the same population as the sample it uses. VaR is not perfect, it can- 
not do everything. But if we understand its flaws — and every professional risk man- 
ager does — then it is a useful guide for day-to-day market risk. If you want to add 
fat tails, fine. But as I will explain below, that is not the solution. 

So, then, why is there so much currency given to a criticism of something that 
no one believes in the first place? 

It is because quant methods sometimes fail. We can quibble with whether ‘some- 
times’ should be replaced with ‘often’ or ‘frequently’ or ‘every now and again,’ but 
we all know they are not perfect. We are not, after all, talking about physics, about 
timeless and universal laws of the universe when we deal with securities. Weird 
stuff happens. And the place where the imperfection is most telling is in risk man- 
agement. 

When the risk manager misses the equivalent of a force five hurricane, we ask 
what is wrong with his methods. By definition, what he missed was a ten or twenty 
standard deviation event, so we tell him he ignored fat tails. There you have it, you 
failed because you did not incorporate fat tails. This is tautological. If I miss a large 
risk — which will occur on occasion even if I am fully competent; that is why they 
are called risks — I will have failed to account for a fat tailed event. I can tell you 
that ahead of time. I can tell you now — as can everyone in risk management — that 
I will miss something. If after the fact you want to castigate me for not incor- 
porating sufficiently fat tailed events, let the flogging begin. 

I remember a cartoon that showed a man sitting behind a desk with a name plate 
that read ‘risk manager.’ The man sitting in front of the desk said, “Be careful? 
That’s all you can tell me, is to be careful?” Observing that extreme events can occur 
in the markets is about as useful as saying “be careful.” We all know they will 
occur. And once they have occurred, we will all kick ourselves and our risk man- 
agers and our models, and ask “how could we have missed that?” 

The flaw comes in the way we answer that question, a question that can be stated 
more analytically as “what are the dynamics of the market that we failed to incor- 
porate.” If we answer by throwing our hands into the air and saying, “well, who 
knows, I guess that was one of them there ten standard deviation events,” or “what 
do you expect; that’s fat tails for you,” we will be in the same place when the next 
crisis arrives. If instead we build our models with fatter and fatter tailed distribu- 
tions, so that after the event we can say, “see, what did I tell you, there was one 
of those fat tailed events that I postulated in my model,” or “see, I told you to be 
careful,” does that count for progress? 
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So, to recap, we all know that there are fat tails; it doesn’t do any good to state 
the mantra over and over again that securities do not follow a Normal distribution. 
Really, we all get it. We should be constructive in trying to move risk management 
beyond the point of simply noting that there are fat tails, beyond admonitions like 
“hey, you know, shit happens, so be careful.” And that means understanding the dy- 
namics that create the fat tails, in particular, that lead to market crisis and unex- 
pected linkages between markets. 

What are these dynamics? 

One of them, which I have written about repeatedly, is the liquidity crisis cycle. 
An exogenous shock occurs in a highly leveraged market, and the resulting forced 
selling leads to a cascading cycle downward in prices. This then propagates to other 
markets as those who need to liquidate find the market that is under pressure no 
longer can support their liquidity needs. Thus there is contagion based not on eco- 
nomic linkages, but based on who is under pressure and what else they are holding. 
This cycle evolves unrelated to historical relationships, out of the reach of VaR-types 
of models, but that does not mean it is beyond analysis. 

Granted it is not easy to trace the risk of these potential liquidity crisis cycles. 
To do so with accuracy, we need to know the leverage and positions of the market 
participants. In my previous post, “Mapping the Market Genome,” I argued that this 
should be the role of a market regulator. But even absent that level of detail, per- 
haps we can get some information indirectly from looking at market flows. 

No doubt there are other dynamics that lead to the fat tailed events currently 
frustrating our efforts to manage risk in the face of market crises. We need to move 
beyond the fat-tail critiques and the ‘be careful’ mantra to discover and analyze 
them. 

The Myth of Non-correlation 

S ee http: 1 1 rick, bookstaber. com 12007 109 1 myth-of-noncorrelation.html 

[This is a modified version of an article I wrote that appeared in the September, 
2007 issue of Institutional Investor.] 

With the collapse of the U.S. sub-prime market and the after-shocks that have 
been felt in credit and equity markets, there has been a lot of talk about fat tails, 
20 standard deviation moves and 100-year event. We seem to hear such descriptions 
fairly frequently, which suggests that maybe all the talk isn’t really about 100-year 
events after all. Maybe it is more a reflection of investors’ market views than it is 
of market reality. 

No market veteran should be surprised to see periods when securities prices move 
violently. The recent rise in credit spreads is nothing compared to what happened 
in 1998 leading up to and following the collapse of hedge fund Long-Term Capital 
Management or, for that matter, during the junk bond crisis earlier that decade, 
when spreads quadrupled. 

What catches many investors off guard and leads them to make the “100 year” 
sort of comment is not the behavior of individual markets, but the concurrent big 
and unexpected moves among markets. It’s the surprising linkages that suddenly 
appear between markets that should not have much to do with one other and the 
failed linkages between those that should march in tandem. That is, investors are 
not as dumbfounded when volatility skyrockets as when correlations go awry. This 
may be because investors depend on correlation for hedging and diversifying. And 
nothing hurts more than to think you are well hedged and then to discover you are 
not hedged at all. 

Surprising Market Linkages 

Correlations between markets, however, can shift wildly and in unanticipated 
ways — and usually at the worst possible time, when there is a crisis with volatility 
that is out of hand. To see this, think back on some of the unexpected correlations 
that have haunted us in earlier market crises: 

• The 1987 stock market crash. During the crash, Wall Street junk bond trading 
desks that had been using Treasury bonds as a hedge were surprised to find 
that their junk bonds tanked while Treasuries strengthened. They had the 
double whammy of losing on the junk bond inventory and on the hedge as 
well. The reason for this is easy to see in retrospect: Investors started to look 
at junk bonds more as stock-like risk than as interest rate vehicles while 
Treasuries became a safe haven during the flight to quality and so were bid 
up. 

• The 1997 Asian crisis. The financial crisis that started in July 1997 with the 
collapse of the Thai baht sank equity markets across Asia and ended up en- 
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veloping Brazil as well. Emerging-markets fund managers who thought they 
had diversified portfolios — and might have inched up their risk accordingly — 
found themselves losing on all fronts. The reason was not that these markets 
had suddenly become economically linked with Brazil, but rather that the 
banks that were in the middle of the crisis, and that were being forced to re- 
duce leverage, could not do so effectively in the illiquid Asian markets, so 
they sold off other assets, including sizable holdings in Brazil. 

• The fall of Long-Term Capital Management in 1998. When the LTCM crisis 
hit, volatility shot up everywhere, as would be expected. Everywhere, that is, 
but Germany. There, the implied volatility dropped to near historical lows. 
Not coincidentally, it was in Germany that LTCM and others had sizable long 
volatility bets; as they closed out of those positions, the derivatives they held 
dropped in price, and the implied volatility thus dropped as well. Chalk one 
up for the adage that markets move to inflict the most pain. 

And now we get to the crazy markets of August 2007. Stresses in a minor part 
of the mortgage market — so minor that Federal Reserve Board Chairman Ben 
Bernanke testified before Congress in March that the impact of the problem had 
been “moderate” — break out not only to affect other mortgages but also to widen 
credit spreads worldwide. And from there, sub-prime somehow links to the equity 
markets. Stock market volatility doubles, the major indexes tumble by 10 percent 
and, most improbable of all, a host of quantitative equity hedge funds — which use 
computer models to try scrupulously to be market neutral — are hit by a “100-year” 
event. 

When we see this sort of thing happening, our not very helpful reaction is to 
shake our heads as if we are looking over a fender bender and point the finger at 
statistical anomalies like fat tails, 100-year events, black swans, or whatever. This 
doesn’t add much to the discourse or to our ultimate understanding. It is just more 
sophisticated ways of saying we just lost a lot of money and were caught by sur- 
prise. Instead of simply stating the obvious, that big and unanticipated events occur, 
we need to try to understand the source of these surprising events. I believe that 
the unexpected shifts in correlation are caused by the same elements I point to in 
my book as the major cause of market crises: complexity and tight coupling. 

Complexity 

Complexity means that an event can propagate in nonlinear and unanticipated 
ways. An example of a complex system from the realm of engineering is the oper- 
ation of a nuclear power plant, where a minor event like a clogged pressure-release 
valve (as occurred at Three Mile Island) or a shift in the combination of steam pro- 
duction and fuel temperature (as at Chernobyl) can cascade into a meltdown. 

For financial markets, complexity is spelled d-e-r-i-v-a-t-i-v-e-s. Many derivatives 
have nonlinear payoffs, so that a small move in the market might lead to a small 
move in the price of the derivative in one instance and to a much larger move in 
the price in another. Many derivatives also lead to unexpected and sometimes un- 
natural linkages between instruments and markets. Thanks to collateralized debt 
obligations, this is what is at the root of the first leg of the contagion we observed 
from the sub-prime market. Sub-primes were included in various CDOs, as were 
other types of mortgages and corporate bonds. Like a kid who brings his cold to a 
birthday party, the sickly sub-prime mortgages mingled with these other instru- 
ments. 

The result can be unexpected higher correlation. Investors that have to reduce 
their derivatives exposure or hedge their exposure by taking positions in the under- 
lying bonds will look at them as part of a CDO. It doesn’t matter if one of the under- 
lying bonds is issued by a AA-rated energy company and another by a BB financial; 
the bonds in a given package will move in lockstep. And although sub-prime hap- 
pens to be the culprit this time around, any one of the markets involved in the CDO 
packaging could have started things off. 

Tight Coupling 

Tight coupling is a term I have borrowed from systems engineering. A tightly cou- 
pled process progresses from one stage to the next with no opportunity to intervene. 
If things are moving out of control, you can’t pull an emergency lever and stop the 
process while a committee convenes to analyze the situation. Examples of tightly 
coupled processes include a space shuttle launch, a nuclear power plant moving to- 
ward criticality and even something as prosaic as bread baking. 

In financial markets tight coupling comes from the feedback between mechanistic 
trading, price changes and subsequent trading based on the price changes. The 
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mechanistic trading can result from a computer-based program or contractual re- 
quirements to reduce leverage when things turn bad. 

In the ’87 crash tight coupling arose from the computer-based trading of those 
running portfolio insurance programs. On Monday, October 19, in response to a 
nearly 10 percent drop in the U.S. market the previous week, these programs trig- 
gered a flood of trades to sell futures to increase the hedge. As those trades hit the 
market, prices dropped, feeding back to the computers, which ordered yet more 
rounds of trading. 

More commonly, tight coupling comes from leverage. When things start to go 
badly for a highly leveraged fund and its collateral drops to the point that it no 
longer has enough assets to meet margin calls, its manager has to start selling as- 
sets. This drops prices, so the collateral declines further, forcing yet more sales. The 
resulting downward cycle is exactly what we saw with the demise of LTCM. 

And it gets worse. Just like complexity, the tight coupling born of leverage can 
lead to surprising linkages between markets. High leverage in one market can end 
up devastating another, unrelated, perfectly healthy market. This happens when a 
market under stress becomes illiquid and fund managers must look to other mar- 
kets: If you can’t sell what you want to sell, you sell what you can. This puts pres- 
sure on markets that have nothing to do with the original problem, other than that 
they happened to be home to securities held by a fund in trouble. Now other highly 
leveraged funds with similar exposure in these markets are forced to sell, and the 
cycle continues. This may be how the sub-prime mess expanded beyond mortgages 
and credit markets to end up stressing quantitative equity hedge funds, funds that 
had nothing to do with sub-prime mortgages. 

All of this means that investors cannot put too much stock in correlations. If you 
depend on diversification or hedges to keep risks under control, then when it mat- 
ters most it may not work. 
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Discussion 

Chairman Miller. Thank you very much. We will now have 
rounds of questions of five minutes for each Member, and I will 
begin by recognizing myself for five minutes. 

Can Economic Events Be Predicted? 

Dr. Bookstaber, what you just described, what I have heard you 
describe as gaming, I have heard celebrated on the Financial Serv- 
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ices Committee, on which I also serve, as innovation — that a lot of 
innovation seems to be simply a way to evade existing regulations. 
And I think both of you got it — Dr. Bookstaber, in that last bit of 
testimony you certainly got at it, but the supporters of the VaR 
now they say want a do-over, that the VaR model was perhaps 
flawed but it can be fixed, and they can now develop a more reli- 
able model that will predict fat tail events, the unlikely events. Do 
you think that it is a failure of that model, or do you think the fail- 
ure is in the idea that economic events can be predicted with the 
same precision that the movement of the planets can be predicted? 
Do you think that it is inherently flawed to think that we can de- 
velop models that will be unfailingly reliable? Dr. Taleb. 

Dr. Taleb. This is my life story. From the beginning — and I 
heard, Dr. Bookstaber and I share a lot of opinions, you know, on 
things like gaming, like the numbers that are going to be gamed 
on the interaction between model and participants. However, there 
are two things or three things that I heavily disagree with, and the 
first one is, he said that we can use different distribution to model 
tail events. Well, that is the story of my life. This is why I provided 
this paper forthcoming in which I look at 20 million pieces of data, 
every single economic variable I could find, and tried to see if there 
is regularity in the data helping to predict itself, you know, outside 
that sample from which it was derived. Unfortunately, it is impos- 
sible, and that is my first argument, that the more remote the 
event, the less we can predict it, and that’s my first point. And the 
second one is, we know which variables are more unpredictable 
than others, and it is very easy to protect against that. And the 
third one is that I agree with Dr. Bookstaber; if I were, you know, 
an omnipotent person seeing all the leverage and everything in the 
system, and equipped with heavy, you know, equations, I could 
probably figure it out. However, this is Soviet-style thinking, that 
someone, some regulator, some unit out there can see what is going 
on and be able to model it, because unfortunately when we model 
in complex systems, we have non-linearity. Even if I gave you all 
the data and you missed something by $1 million, okay — your prob- 
abilities will change markedly. 

Chairman Miller. I will get to you, Dr. Bookstaber, but your so- 
lution then is just higher liquidity requirements? 

Dr. Taleb. No, my solution is figuring out — it is very simple. I 
was a trader in the 1980s. There were some products we could real- 
ly risk manage on a napkin. Options, instruments, futures, all 
these we could risk manage on a napkin. Once we started having 
these toxic products — to me, the sole purpose of these products is 
to create bonuses, like complex derivatives. I was a complex deriva- 
tives trader. I have a textbook on complex derivatives, and I tell 
you, these products, okay, can hide massive amounts of tail risks. 
They are not needed for anyone. A lot of these products should not 
be there. If you eliminate some of the products, some of the expo- 
sure, it would not change anything to economic life and it would 
make things a lot more measurable. So my solution is to ban some 
products that have a toxic exposure to tail events. 

Chairman Miller. Dr. Bookstaber. 

Dr. Bookstaber. Let me just correct one point. I do not advocate 
trying to fix VaR by fattening the tails. I am simply arguing that 
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some people make that as a suggestion. I think VaR is what it is, 
it does what it does, and the best thing to do is recognize the limi- 
tations of VaR, which I stated, and use it for what it is good for 
but not try to oversell it, not to think that it represents all possible 
risk, because any attempts to somehow make it more sophisticated 
are just going to obfuscate it all the more. So you take VaR as one 
tool for risk management, and then extend out from there. 

The second point, just addressing what you are saying, is that, 
number one, I don’t think that you can use VaR and have a ‘do- 
over’ to try to expand it and have it solve these crisis-type prob- 
lems. I also don’t think that we will ever be at the point of being 
able to know all the risks. But I do think that we can move some- 
what in the direction of understanding crisis risk more. But to do 
it, you need the data, and the data that you really need to start 
with is: how highly leveraged are the people in the market, and 
what are their positions — so that if there is a shock in a particular 
market, will there be so much leverage there that people will be 
forced to liquidate? What other positions do they have, so how 
could that propagate out? It is not a panacea. You can’t have a sil- 
ver bullet because of the feedback and gaming capabilities but I 
think you can move more in the direction of dealing with these cri- 
sis risks. 


Regulation of Financial Products 

Chairman Miller. My time has expired but I have a question 
that is sort of in hot pursuit of what you both just said, and I will 
be similarly indulgent to the other Members here. 

Dr. Taleb, you said there should be something like a Food and 
Drug Administration (FDA) to look at financial products, to see if 
they actually do something useful, or if they simply create addi- 
tional risks that create short-term profits. Apparently about 90 per- 
cent of derivatives — I was only half kidding when I asked you if 
your mother knew you designed derivatives. But in about 90 per- 
cent of derivatives, no party to the transaction has any interest in 
the underlying, whatever it was, that the derivative is derived 
from — credit default swaps. Do you agree that some financial prod- 
ucts should simply be banned as having no readily discernible use- 
fulness, utility for society, for the economy — and creating a risk 
that we cannot begin to understand? Should credit default swaps 
be banned? Should they be limited to — have a requirement that is 
equivalent to an insurable interest requirement in insurance law? 
Dr. Taleb or Dr. Bookstaber? 

Dr. Taleb. I cannot — I don’t — I am not into regulation to know 
whether we should be allowed to ban people based on uses but — 
based on risk, okay, because society doesn’t bear the risk. I have 
here what I call the ‘fourth quadrant,’ and we should ban financial 
products — and when I call it the fourth, it is a little technical, but 
it is a very simple rule of thumb that takes minutes to check if a 
given financial product belongs or doesn’t belong to the fourth 
quadrant. In other words, does it have any explosive toxic effects 
on either the user or the issuer, or both, you know, so it is very 
easy. So these products — and this is how I have my fourth quad- 
rant — these are the exposures we should not just compute, you 
know, but eliminate. And there are a lot of things we can measure. 
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I mean, I may agree with Dr. Bookstaber, VaR may work for some 
products, and we know which ones, but not for these products that 
have open-ended, toxic, geometric — what I call geometric — in other 
words, escalating payoffs. 

Chairman Miller. Dr. Bookstaber. 

Dr. Bookstaber. For reference, I refer the Committee to testi- 
mony that I gave in June to the Agricultural Committee of the Sen- 
ate on the topic of derivatives, and there I pointed out that, over 
time, derivatives have moved more and more towards being used 
for gaming. In fact, I said that derivatives are the weapon of choice 
for gaming the system. They are used to allow you to hedge when 
you are not supposed to hedge, to avoid taxes, to lever when you 
are not supposed to lever. There is vested interest on both the sell 
and the buy side to have derivatives that are complex and obfus- 
cating, that are customized. I believe, number one, that many de- 
rivative instruments that exist today are used more for either gam- 
ing or gambling purposes as opposed to having true economic func- 
tion. And I believe that there are many customized and complex in- 
struments that could easily be transformed into a set of standard- 
ized instruments that would be easier to track, more transparent, 
and possibly even put on an exchange. So I certainly agree with the 
concept that derivatives is a point to focus on, because it is one of 
the ways that we find risk coming in these tail events in surprising 
ways. 

Chairman Miller. Thank you, Dr. Bookstaber. 

I now recognize Dr. Broun for nine minutes and 45 seconds. 

‘Too Big to Fail’? 

Mr. Broun. Thank you, Mr. Chairman. I want to make a quick 
statement. I believe, first thing, that there is no such thing as an 
entity that is too big to fail, particularly when we look at busi- 
nesses, even large businesses such as the investment banks, and 
I believe in holding people personally accountable and responsible, 
and I believe that when you take away the taxpayer safety net that 
people are utilizing to gamble away other people’s future, then peo- 
ple will be held more accountable and will make better decisions. 
I think greed and lust are two tremendous blinding factors when 
people start making decisions. 

Having said that, I also want to state that I think that there 
were a lot of warning signs about this current economic crisis that 
we found ourselves in, and many people sounded the horn of warn- 
ing saying that we needed to change federal law and regulation to 
prevent what has happened, and those warnings were unheeded by 
Congress and by people who were in the decision-making process. 
Having said that, I am real concerned too because investment 
banks took excessive risk based on these models and commercial 
banks are also now forced to rein in risk, even though they are not 
taking risky positions to begin with, those commercial banks. What 
can we do to ensure that small commercial banks around the coun- 
try are not punished by the risky behavior of large investment 
banks? Either or both, who wants to go first? 

Dr. Bookstaber. That is a difficult question, and I don’t know 
that I can illuminate it too much, but I can go in a particular direc- 
tion. You can correct me if I am going the wrong way. I think there 
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is a distinction between the larger banks, which de facto actually 
are the investment banks, and the smaller banks, because the larg- 
er banks end up quasi-market makers in the sense that they take 
on positions of risk for clients. They become market makers in the 
fixed-income market. They issue and support derivatives. They also 
have proprietary trading desks so they are also quasi-hedge funds. 
So I think you can look at the various functions of banks, and look 
at smaller banks, and they typically have a pure banking function. 
Larger banks are not really just bigger versions of smaller banks. 
They are actually institutions that take different types of risk that 
smaller banks don’t take, that can have some of these tail events 
of their own creation — that are demons of their own design — that 
they have created because they have elected to go into the deriva- 
tives markets, or take market-making functions. 

So I think the question for a regulator is, do you have a different 
set of regulations and requirements for the banks that — it is not 
an issue of being too big to fail, but banks that are taking on types 
of risk that make them distinct from their smaller cousins. 

Mr. Broun. Isn’t it greed that drives that as far as the large in- 
stitutions, though? 

Dr. Bookstaber. Well, you know, greed has a little bit of spin 
to it. I mean, there are incentives, and people act based on their 
incentives; and if we give somebody a set of incentives that, as Dr. 
Taleb has mentioned, lead them to say, ‘I want to take risks which 
might blow the bank up, with small probability, but with very high 
probability will give me a large bonus,’ you are going to have peo- 
ple acting accordingly. So I think the way to think of it is, not that 
they are acting on the basis of greed, but they are acting on the 
basis of incentives that lead to behavior that, for the market over- 
all, may be unduly risky. 

Mr. Broun. Isn’t it so particularly when you have somebody else 
who is going to be held responsible for that decision-making proc- 
ess? 

Dr. Bookstaber. Right. 

Mr. Broun. Like the taxpayer is going to be on the hook if they 
make a bad decision. 

Dr. Bookstaber. That is right. There is no doubt that incentives 
have played a large role in what we have observed. You know, had 
you had, for example, somebody like Mr. Prince saying — apparently 
recognizing the riskiness of what they are doing — and saying, well, 
as long as the music is playing, we are going to keep dancing. Why 
is he going to keep dancing? Because his incentive is based on next 
quarter’s earnings, and he can’t walk away from that dance floor 
while his competitors are still on it, because his incentives are 
structured to make that incorrect decision. 

Mr. Broun. And he has everything to gain and nothing to lose 
in that process, correct? 

Dr. Bookstaber. Yes. 

Mr. Broun. Dr. Taleb. 

Dr. Taleb. Yes. Well, I just wrote a paper with my colleague 
(Charles Tapiero) in which we showed why — I don’t know if you 
have heard about the case of Societe Generale, the French bank 
that lost $7 billion, $8 billion on a rogue trader, and we showed 
that it came from too big a size. Size has effect in compounding 
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risk, and let me give you the intuition. If you have a bank a tenth 
of the size of Societe Generate, and they had a rogue trader that 
had a tenth of the size of the position of that rogue trader, the 
losses would have been close to zero. The fact that they had to liq- 
uidate, they discovered that that rogue trader had 50 billion euros 
in hidden position and they had to liquidate that, and liquidating 
50 billion euros rapidly costs a lot more than liquidating five billion 
euros. You liquidate five billion euros at no transaction cost almost, 
or a very small transaction cost, compared to liquidating 50 billion. 
So that would generalize to risks of unexpected events tend to af- 
fect large size more. 

And I have here another comment to make about banks. Banks, 
of course, have done so far — I mean, we have evidence they have 
done, so far, very little for society, except generate bonuses for 
themselves, from the data, and that is not from recent events that 
I am deriving that. When I wrote “The Black Swan” it was before 
these events. But look at hedge funds. Hedge funds, I heard the 
number, 1,800 hedge fund failed in the last episode. I don’t know 
if many of them made the front page of any Washington paper. So 
the hedge funds seem to be taking risks without endangering soci- 
ety, or at least not taxpayers directly. And this model of hedge fund 
corresponds to my norm, okay — what is a complex system that is 
robust? The best one is Mother Nature. Mother Nature has a lot 
of interdependence. We have an ecosystem, a lot of interdepend- 
ence. But if you went and shot the largest mammal, a whale, or 
the largest land mammal, an elephant, you would not destroy the 
ecosystem. If you shot Lehman Brothers, well, you know what hap- 
pened, okay. You destroyed the system — too much interdependence 
means you should not have large units. But hedge funds have 
shown us the way to go. They are born and they die every day, lit- 
erally every day. Today I am sure that many hedge funds are born 
and many hedge funds have died. So this is a model that replicates 
how nature works with interdependence. But of course we have to 
listen to Dr. Bookstaber’s advice to make sure that they don’t all 
have the same positions you have to put the exclusionary system, 
but they have a lot more diversity than banks. 

Walt. Street’s Dependency on Government Bailouts 

Mr. Broun. Isn’t it though that the implied or even outright 
safety net of the taxpayers picking up the pieces if there is a fail- 
ure, isn’t that the thing that is driving the derivatives and all these 
other complex financial instruments that cause people to make 
these risky behavior judgments? 

Dr. Taleb. Well, I am under oath and I will say exactly some- 
thing that I want to be on the record. I was a trader for 21 years, 
and every time I said what if we blow up, he said, who cares, the 
government bails us out. And I heard that so many times through- 
out my career, that, “don’t worry about extreme risks, worry about 
down five percent, ten percent, don’t worry about extreme risks, 
they are not your problem anymore, it is not our problem.” I heard 
that so many times, and here I am under oath and I say it. 

Dr. Bookstaber. If I may add to that, there is the notion, well 
known, of what is called the trader’s option. The trader’s option is, 
I get X percent of the upside and limited or zero of the downside, 
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but that trader’s option extends also in many cases to the manage- 
ment of the firms. They get the upside and so you would much 
rather, you know, construct a position that makes a little, makes 
a little, makes a little and makes a little and with small probability 
loses everything, because that increases the chance that you have 
consistent earnings, consistent bonuses, and in the extreme events, 
your downside is limited because of the option characteristic of 
your compensation. 

Mr. Broun. So in the ten seconds I have left, I just want to state 
that taking away the government safety net is going to make peo- 
ple more responsible and they will make better decisions on a real 
risk management basis, and I thank you all. It is my opinion that 
that is what I am getting from you all, correct? 

Dr. Taleb. In my opinion as well. 

Dr. Bookstaber. If I may, I would just say, it is not just the 
safety net. If I am an individual in a firm, I don’t care about the 
safety net, I care about my own bonus, so with or without the safe- 
ty net for the firm overall, if my incentives are, I make money if 
things go up, I get a new job if things blow up, I don’t know that 
the safety net matters to me personally. 

Dr. Taleb. May I respond to this point? 

Chairman Miller. Dr. Taleb. 

Dr. Taleb. I agree that if I am a trader, I don’t care who is going 
to bail me out. The problem is that the shareholders don’t care 
when society can bail them out because there is unlimited liability, 
that shareholders are protected so society bears the rest. So we 
have three layers: a trader, the shareholder and thirdly, society. So 
in the end, the free option comes from society. 

Mr. Broun. Thank you, Mr. Chairman. 

Chairman Miller. Thank you. 

I think something like 90 percent of American households have 
a household income of less than $105,000 a year, so for a trader 
to make $100 million, $120 million does not seem like make a lit- 
tle, lose a lot. 

Ms. Dahlkemper for five minutes. 

Ms. Dahlkemper. Thank you, Mr. Chairman. 

I wanted to go back to your statement in terms of some — that 
maybe some financial products should be banned, and there are 
some that may argue that banning any financial product is an ex- 
cessive intrusion into the free market. So if you could just give me 
your response to that claim. 

Dr. Taleb. I believe in free markets but I do not believe in state 
socialism, okay, and I don’t believe — I believe the situation we have 
had so far is not free markets. It is socialism for losses and cap- 
italism for profits. So if the taxpayer is involved ultimately in bail- 
ing out, which the taxpayer should be able to say, I want this prod- 
uct or that product, the risk, okay? You know, my opinion, I am 
in favor of free markets but that is not my definition of free mar- 
kets, okay, state-sponsored socialism for the losses and capitalism 
for the profit — I mean, free market for the profit. That I don’t — as 
a taxpayer, and I am paying taxes. 

Ms. Dahlkemper. Dr. Bookstaber. 

Dr. Bookstaber. I think even in a capitalist system, the argu- 
ment that some products should not go forward or should be 
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banned is a reasonable one for the following reason: that if I con- 
struct some new product, and let us say it is a fairly complex prod- 
uct or has a fat tail and it can inflict problems for society, there 
is a negative externality to that product that is not priced — that is, 
I sell it, I create it, somebody wants to buy it, but the negative ex- 
ternality is the increased probability of crisis that it causes, and 
any time that you have a non-price-negative externality is a time 
that I think even a libertarian would argue you can have govern- 
ment intervention. 

The Risks of Different Tupes of Institutions 

Ms. Dahlkemper. Thank you. I also wanted to go back a little 
bit to the ‘too big to fail’ subject in terms of the institutions. When 
we look at the surviving large banks, they are bigger than ever, so 
where do you know when an institution is ‘too big to fail’ and how 
do we restructure these firms? 

Dr. Taleb. ‘Too big to fail,’ you can see it. If anything in nature 
is bigger than an elephant, it won’t survive, and you can see, I am 
sure anything bigger than a large hedge fund, to me, is too big. But 
there is one thing here associated with the problem. The reason we 
depend so much on banks is because the economy has been over- 
financialized over the past 25 years, over-financialized. The level of 
debt today in relation to GDP is three times, according to some 
numbers, even more or less, but three times the level of debt to 
GDP that we had in the 1980s. So that is rather worrisome. This 
is why we have ‘too big’ banks, all right, because it comes with the 
system. It is a process, you know, that feeds on itself, that is a re- 
cursive process. And if we definancialize the economy more, the 
debt level will come down. Then the discussion about ‘too big to 
fail,’ about banks, will be less relevant. I mean, banks’ role is not 
so — you know, banks where I can withdraw money when, you 
know, when I go to Atlanta and then there is a bank that is used 
for letter of credit, very useful things for society. And there are 
banks that trade for speculative reason, banks that issue paper 
that nobody needs and there are banks, the banking that cor- 
responds to lending, you know, increased lending because a lender 
makes a bonus based on the size of loans. So if you brought this 
down, the size of banks would drop dramatically. Particularly, the 
balance sheets would shrink dramatically, and particularly if we 
moved the risk away from banks. The banks are more of a utility 
in the end, and they are hijacking us because a utility with a bonus 
structure, it doesn’t work. As I said here, don’t give someone man- 
aging a nuclear plant a bonus based on cost savings, okay, or based 
on profitability. You don’t, all right? So the problem is, they are hi- 
jacking us because of the dual function of a utility that we need 
them to have, a letter of credit or something as basic as with- 
drawing cash, and at the same time they take risks with bonuses. 
So if we brought down the level of banking, moved the risks more 
and more to hedge funds, these people are adults, they don’t endan- 
ger anyone, just make sure they don’t get big and have Dr. 
Bookstaber’s rules on, you know, leverage and stuff like that well 
enforced . . . then the level of — then that problem would dis- 
appear. So let us worry more about the cancer rather than worry 
about the symptoms. 
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Ms. Dahlkemper. Dr. Bookstaber. 

Dr. Bookstaber. You know, the Treasury came out with some 
principles for regulation of capital on September 3, and one of the 
key issues that they mentioned is dealing with ‘too big to fail.’ I 
think one of the difficulties is, I don’t think we can measure too big 
to fail. I don’t think we know. It is not just a matter of the capital 
that you have or the leverage that you have. For example, LTCM 
was a hedge fund and it was a relatively small firm and had $3 
billion capital, yet in a sense it was ‘too big to fail’ because it al- 
most brought down, actually, Lehman along with it, and the Fed 
had to step in. What matters is how what you are doing weaves 
in with what other people, what other funds or firms are doing 
within the economy. So you could have a ‘too big to fail’ that is not 
predicated on one institution and what that institution is doing, 
but it could be based on some strategy or some new instrument, 
where for anyone from that instrument that strategy is relatively 
small, but if the exogenous shock occurs in the market and it af- 
fects that strategy, it affects so many firms in the same way that 
it has a substantial systemic effect. And I get back to the point that 
we don’t have the information to even know right now what type 
of positions or leverage or strategies might have that threading 
across different institutions. 

Ms. Dahlkemper. Thank you. My time is expired. 

Chairman Miller. We are about to have 40 minutes of votes 
shortly so I would like for both Mr. Wilson and Mr. Grayson to 
have a chance to ask questions. I should just tell the panel that 
this Charlie Wilson has never had a movie made about him. So far 
as I know, he has never been in a hot tub. Mr. Wilson for five min- 
utes. 


Incentive Structures for Trades 

Mr. Wilson. Thank you, Mr. Chairman. 

Gentlemen, good morning. I serve on the Financial Services Com- 
mittee also and I have to keep pinching myself that really I am in 
a Science and Technology Subcommittee and so it is hard to realize 
the conversations we are having. Dr. Taleb, if I could say that, you 
know, what you said earlier in your testimony about people not 
being concerned about the success or failure of a firm because they 
knew there would be a public bailout is frightening. That is cer- 
tainly not the American way or certainly not the way we want to 
do business. With those things in mind, I have a couple questions 
I would like to ask and maybe we can get some of your feeling as 
to how people would get so far off track, that that would be the 
thought process. That concerns me. 

People have been outraged at the size of the bonuses and espe- 
cially when we were doing the voting for the bailout. Some of the 
employees were bailed out, as you all know, with government 
money, huge amounts of money to the Wall Street firms. Much of 
the conversation was about firms being ‘too big to fail,’ and you say 
that in the bonuses, that is really the motivator for everybody. I 
would hate to think that there was no leadership that wouldn’t try 
to keep people on the right track rather than money being the only 
motivator, the true part of it. So can you explain that? And I was 
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going to address this question, if I could, to Dr. Bookstaber if I 
could. 

Dr. Taleb. You would like me to explain how people were han- 
dling extreme risks. 

Mr. Wilson. I did. That was confusing. I am sorry. I did address 
that to you but I would be interested in Dr. Bookstaber also. If you 
would go first, Dr. Bookstaber, please. 

Dr. Bookstaber. Thank you. I don’t think — I don’t mean to be 
cynical, but I don’t think that leadership within a financial firm 
can overcome the incentives that exist, incentives not just including 
the trader’s option, but to do the bidding of the people who have 
put you in your position, namely the shareholders whose interest 
is earnings and maybe even earnings quarter by quarter. So I 
think the way that you have to change things is through the incen- 
tive structure of the people who are taking risk in ways that has 
been widely discussed, and I think it is fairly clear that you don’t 
finally get paid until whatever trade you have put on, or whatever 
position you put on, is off the books and has been recorded. You 
can’t basically put on positions and get paid based on the flow of 
those positions until the trade is realized, that is, until the book 
is closed on that trade. So this is the notion of longer-term incen- 
tives. So if you have longer-term incentives, if you have incentives 
where you can’t game the system by constructing trades or port- 
folios that again make a little, make a little, maybe blow up, then 
people will act based on those incentives. But the leadership of the 
firm is always going to have the following statement, that our re- 
sponsibility is to the shareholders, we have to maximize share- 
holder value, and then the shareholders, by the way, although in 
theory they have a vote, in practice don’t. And so, you know, you 
have sort of this — the management pointing towards the share- 
holders, the shareholders effectively being silent partners within 
the corporation. 

Mr. Wilson. Thank you. 

Dr. Talber, am I saying that right? 

Dr. Taleb. Taleb. 

Mr. Wilson. Taleb. I am sorry. 

Dr. Taleb. There are two problems, and I gave two names, a 
name to each problem. The first one is called, the title of my first 
book, fools of randomness, ‘Fooled by Randomness,’ and other peo- 
ple who believe their own story and actually don’t know that they 
are engaging in these huge amount of hidden risks out of psycho- 
logical, you know — as humans, we are not good at seeing negative 
outcomes. We are victims of overconfidence, so we make these mis- 
takes whether or not there is a bonus, is the psychological, the first 
one. And the second one, I call them ‘crooks of randomness,’ so 
there is ‘fools of randomness’ and ‘crooks of randomness,’ and you 
always have the presence of both ailments in a system. Like, for 
example, when we had LTCM, Long-Term Capital Management, 
the problem, these people had their money in it, so, visibly, they 
were not gaming the system consciously, all right, they were just 
incapable of understanding that securities can take large vari- 
ations. So there are these two problems. So the bonus, it is impera- 
tive to fix the bonus structure, and as I said here, that I don’t know 
any place in society where people manage risk and get a bonus. 
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The military people, the police, they don’t get a bonus. So fix, make 
sure that he who bears risk for society doesn’t have a bonus. Fix 
the bonus structure that is not sufficient. 

Mr. Wilson. One of the things that, you know, we have heard 
a lot about since the money was invested in Wall Street was that 
if the big bonuses didn’t continue, the firms couldn’t necessarily 
keep the talent. Do you have any comment on that, Dr. Taleb? 

Dr. Taleb. I am laughing, sorry, because a lot of these people — 
in my book there is a gentleman who had $10 million, a big hotshot 
trader, and when he blew up, he couldn’t even drive a car. I mean, 
you can find better cab drivers. I don’t know what you could do 
with these Wall Street derivatives, high-income people other than 
use them as drivers but even then, I mean, you can use someone 
less reckless as a driver. So I don’t know what to use them for, 
honestly. I don’t know what is the — I was on Wall Street for 21 
years and a lot of people I wouldn’t use for anything. I don’t know 
if you have some suggestions. So I don’t know what you are com- 
peting against, all right, and you have high unemployment on Wall 
Street, and calling that ‘talent’ is a real — it is a very strange use 
of language, people who lost $4.3 trillion worldwide in the profes- 
sion, and then calling it ‘talent.’ So there is talent in generating bo- 
nuses, definitely, that you cannot deny. Other than that, I don’t 
know. 

Dr. Bookstaber. There is — on this point, there are people who 
are not merely talented, but gifted, in areas like medicine and 
physics and other fields and they seem to get by on some amount 
of money, $200,000, $500,000, $1,000,000. I don’t know that the 
talent in Wall Street is so stellar that it is worth $50 million or 
$100 million versus the talent in these other fields. The issue with 
the talent more is that the structure of Wall Street somehow allows 
that level of compensation, so if one firm does not allow it, people 
can move to another firm that does. But if there is a leveling of 
the field overall so that instead of $20 million people are making 
$1 million or $2 million, you know, then I think this issue of, you 
know, ‘we will lose our talent’ disappears. It has to be done in a 
uniform way, as opposed to affecting just one firm versus another. 

Mr. Wilson. Thank you. 

Chairman Miller. Dr. Taleb, do you want to respond? 

Dr. Taleb. Yes, I have one comment. He is making a socialistic 
argument to limit bonuses. I am making a capitalistic argument to 
limit bonuses. I am saying if people want to pay each other, they 
can pay whatever they want. I just don’t want society to subsidize 
bonuses. That is it. I am making the opposite argument coming 
from — so this is an extreme bipartisan conclusion here where 

Mr. Wilson. We have a few of those here. 

Dr. Taleb. If people want to take risk, you know, and two adults 
can hurt each other financially as much as they want. The problem 
is, as a taxpayer, okay, I don’t want these bonuses. 

Mr. WILSON. Thank you. Thank you both. 

Mr. Chairman, just one comment if I could. It just seems that we 
have to try to find a way to legislate maybe some character to Wall 
Street. 

Chairman Miller. Thank you. I misread the note that said that 
we would shortly have 40 minutes of votes. We will have votes at 
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around 11:45 and they will last 40 minutes, so I am delighted that 
we will be able to continue with this panel for Mr. Grayson and for 
a second round of questioning. Mr. Grayson. 

Holding Wall Street Accountable for Bonuses 

Mr. Grayson. Thank you, Mr. Chairman. 

We are talking today about what proper incentive structures we 
should have on Wall Street, and I am wondering if we are talking 
too much about carrots and not enough about sticks. In fact, people 
on Wall Street did lose over $4 trillion of our money, and I have 
seen almost no one punished for it. Don’t you think that it would 
be likely to deter bad behavior and an overly fond view of risks if 
we actually punished people? 

Dr. Taleb. I am not a legal scholar but there has got to be a way 
to — there is something called malpractice, okay. There has got to 
be a way where we can go after these people that I haven’t seen 
so far, because people are scared, because Wall Street has ‘talents.’ 
These people would run away and go to Monte Carlo or something, 
so we are afraid of letting them, you know, of them running away, 
but we should be doing it immediately, find people who made 
[these losses] — like the Chairman of an executive committee or the 
firm that we had to support who made $120 million of bonuses, and 
supervised unfettered risk taking and made sure that that gen- 
tleman got returns of $120 million bonuses. The place where my 
idea was most popular was Switzerland. The first event of a 
clawback in any country took place in Switzerland, where the au- 
thorities went to Mr. Marcel Ospel, head of UBS, after the events 
of October and told him, listen, give us 12 million Swiss francs, 
please, and it was voluntary and he gave back almost — a large 
share of his — but he clawed back his bonuses. 

Mr. Grayson. But it was voluntary only because the government 
intervenes by limiting people’s liability. The concept of liability is 
determined by our law, not by the free market. In fact, if we were 
to say that we will not give people the right to hide behind cor- 
porate shields, wouldn’t that have a dramatic effect on holding peo- 
ple accountable for the bad decisions that they make? 

Dr. Taleb. To answer, okay, this is still the same problem, fooled 
by randomness or not fooled by randomness. Some people I have 
seen in Chicago trade their own money and lose huge amounts of 
money, not knowing they could lose it, so someone whose net worth 
is $2 million loses $2 million and had to go burn his house to col- 
lect insurance money. So I have seen that. It is not just — so people 
sometimes engage in crazy trades, okay, where they have liability 
themselves. It may not be sufficient, but it would be, for me, eco- 
nomically, a good way to have a bonus compensated by malice be- 
cause capitalism is not just about incentives, it is about punish- 
ment. 

Mr. Grayson. When you say it wouldn’t be sufficient, all you are 
really saying is that it wouldn’t solve the problem for all time, for- 
ever in every case, but it would certainly be a step in the right di- 
rection. 

Dr. Taleb. Oh, it would be imperative, not a step. 

Mr. Grayson. Imperative? 

Dr. Taleb. It is an imperative. 
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Mr. Grayson. Okay. Now, Dr. Bookstaber, I understand that in 
Sweden, the bank managers have unlimited liability for the mis- 
takes that they make, but what happened in our system with re- 
gard to blow-ups, with regard to crazy risks that people take in 
order to pad their own pockets, what effect would that have if we 
were to take that law and introduce it in America? 

Malpractice in Risk Management 

Dr. Bookstaber. You know, something along those lines that I 
have advocated is to have the potential of penalties for the risk 
managers within a firm similar to what are there for the CFO of 
a firm. You know, if a CFO knowingly allows some accounting 
statement to go out, where he knows it is incorrect, he is on the 
hook not just from a civil but from a criminal standpoint. If you 
had the risk managers have to sign on the dotted line, that the 
risk — that they have executed their function correctly, and all ma- 
terial risks have been duly represented — I think that could go a 
long way towards solving the problem, because they would then 
have an incentive to make sure everything is right. And there are 
cases, I think, as we go back to this last crisis, where the risk man- 
agers were in some sense not up to the task, or possibly in bed 
with the people involved in trading or with senior management, to 
where they were willing to have their views overridden — because 
they had no liability on the one side, and they didn’t want to get 
fired on the other. 

Mr. Grayson. But don’t we have to do more than that? Don’t we 
have to not only say to people, you have to fill out these forms 
properly and you have to disclose, but we have to actually hold peo- 
ple accountable for the mistakes that they make, and hold them 
personally accountable? Isn’t that what we need to actually deter 
this kind of misconduct? 

Dr. Bookstaber. I guess the question is what type of mistake, 
because everybody makes certain types of mistakes. I think that 
sort of mistake where you can hold people accountable is where 
they — obviously if they knowingly misrepresent — but where there 
is something material that they — on the one hand it is a mal- 
practice where you say, you know, somebody doing this job in a 
reasonable way should have discovered that. 

Mr. Grayson. But let us talk about the specific problems we 
have seen time and time again in the last few years. Let us talk 
about, for instance, AIG. In AIG, the fundamental problem is that 
the traders entered into literally billions upon billions of dollars of 
heads, I win, tails, you lose bets, bets that couldn’t possibly be 
made good on by anybody but the U.S. Government, and that 
wasn’t a problem of not filling out the form properly, not disclosing. 
Don’t those people need to be punished in order to deter that con- 
duct in the future? 

Dr. Bookstaber. Well, this gets to Dr. Taleb’s point that you 
would have to go into the mindset of the people. Was it, as he is 
saying, you know 

Dr. Taleb. Crooks or fools. 

Dr. Bookstaber. Yeah, were they crooks or fools. If you can dis- 
cern one from the other, then I agree with you, but what I am say- 
ing is, you could also go one level higher to require, which now is 
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required, risk management oversight for those functions where it 
is believed to be credible, and these were supposed to be the people 
who know how to do their job, and they have the responsibility to 
represent that this type of event is not occurring. 

Mr. Grayson. Dr. Taleb. 

Dr. Taleb. Yes. Well, the problem I saw and I wrote about, actu- 
ally, in one of my writings not yet published, I say it is easier to 
fool a million than fool a person and it is much easier to fool people 
with billions than to fool them with millions. Why? Because you 
have bandwagon effects, and you have collective — something called 
diffusion of collective responsibility, and I will tell you exactly why. 
If you have — what risk managers are doing is to make sure they 
do exactly what other risk managers do. If there is a mistake, it 
is a mistake that they did not commit individually, but com- 
mitted — that had company on that. We call it ‘company on a trade.’ 
It is not like an individual doctor who is just incompetent. It is col- 
lective incompetence. We had collective risk management incom- 
petence, but they were all doing what other people — the hedge is 
to do what the other guy is doing and that, I don’t know if, you 
know 

Chairman Miller. Well, the note I got earlier was incorrect and 
now it appears we are going to have votes at any moment, so I will 
start a round of questions and we will try to keep it — I know that 
everybody would like to ask questions of this panel. 

Clawback Provisions 

Just one — it is not clear to me whether you actually supported 
a legal requirement that there be clawback provisions in bonus con- 
tracts, that if a bonus is based upon a profit this year, that if the 
very same transaction results in a loss in two or three years there 
be requirement, a legal requirement that that bonus be repaid. Dr. 
Taleb? 

Dr. Taleb. Indeed. 

Chairman Miller. You do 

Dr. Taleb. Indeed. 

Chairman Miller. Dr. Bookstaber. 

Dr. Bookstaber. I don’t know that I would go to the extent of 
having it be a legal requirement. Ideally, it should be requirements 
placed on the corporation by the equity holders, because it makes 
good economic sense. I think the issue of it being a legal require- 
ment gets into the question of, okay, if we are ultimately the ones 
holding the bag if this fails, we now have a societal obligation. But 
I think whether it is done through the shareholders or if it is legis- 
lated, that type of structure, incentive structure, clearly makes 
sense for trading. 

Chairman Miller. Dr. Taleb. 

Dr. Taleb. There is an additional problem other than the 
clawback. There is the fact that if in any given year, I take $1 mil- 
lion from you, okay — say I win, I get my bonus, and I lose, you 
keep all the losses, so that clawback situation doesn’t solve the free 
option problem. You are solving the mistiming problem, you are not 
solving the free option problem. 

So we have two problems with bonuses. The first one is sym- 
metry. In other words, I make, all right, either a big bonus or noth- 
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ing, whereas if he loses, I take his money, risk his money. He loses 
or makes [money], all right, I just make [money], I just earn. So 
that problem is not solved with the clawback. For example, say the 
TARP money we gave Goldman, all right — okay, let us forget about 
clawbacks. Had they lost money, all right, it would have been — we 
would have eaten the loss. If they made money, they kept the bo- 
nuses, okay, so that idea of having just profits and never losses, 
net, net . . . the clawback is about repaying previous bonuses, but 
it doesn’t address the vicious incentive of paying someone for the 
profits and not charging him for the overall losses, and the 
clawback doesn’t solve that. 

Chairman Miller. Are you suggesting that that should be pro- 
hibited by law, or should people just have better sense than to 
agree to that kind of compensation system? 

Dr. Taleb. In other words, people should have skin in the game. 
Net, net, net, if I fail, I should be penalized personally some way 
or another. Don’t have an option where I only have the profits and 
none of the losses. 

Chairman Miller. I am still not clear if you are suggesting that 
that be a legal requirement or there simply should be a change in 
the culture, that anyone who agrees to a hedge fund compensation 
of 220 is a fool, and if people stopped agreeing to it, the compensa- 
tion system would change. 

Dr. Taleb. No, to me, it should be only a legal requirement 
wherever TARP or a possible society bailout is possible. If there is 
no society — if someone signs no society bailout, then no. 

Credit Default Swaps 

Chairman Miller. I asked the question earlier but I am not sure 
I got a clear answer. Do you think credit default swaps should be 
banned? If not, do you think they should be limited to — they should 
have a requirement that would be comparable to the requirement 
of an insurable interest in insurance law? 

Dr. Bookstaber. I agree with the latter. I don’t believe that 
credit default swaps should be banned, because they do have eco- 
nomic function in the sense that — if I have the debt of a company 
and perhaps it is illegal, or for some reason it is difficult for me 
to undo my risk by selling it, I can use the swap to mitigate or 
hedge my risk. But I don’t think that it should turn into what it 
has turned into — basically, a gambling parlor of side bets for people 
who have no economic interest at all in the underlying firm. The 
point you mentioned, Mr. Chairman, in your opening remarks, that 
the number of people doing side bets far exceeds those who actually 
have an economic reason to be taking that exposure. 

Chairman Miller. Dr. Taleb. 

Dr. Taleb. Mr. Chairman, these products are absurd. They are 
class B products for me, for the simple reason that it is like some- 
one buying insurance on the Titanic from someone on the Titanic. 
These credit default swaps, you buy them from a bank, so they 
make no sense. And I have been writing about these class B instru- 
ments that have absolutely no meaning and I don’t believe that 
they have economic justification other than [to] generate bonuses. 

Chairman Miller. The other analogy I have heard is buying in- 
surance against a nuclear holocaust; if you think you are going to 
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be around to file a claim, who do you think you are going to file 
it with. I will give up my own time; Dr. Broun. 

Were the Bailouts and Stimulus Funds Necessary? 

Mr. Broun. Thank you, Mr. Chairman. Do you believe that bail- 
ing out banks and transferring debt from private sources to public 
sources is a responsible action? 

Dr. Taleb. I mean, my opinion is, I am going to be very, very, 
very honest — it is irresponsible because we have levels of about $60 
trillion, $70 trillion worldwide in excess debt that is being slowly 
transformed into something for our children. If a company goes 
bankrupt, that debt disappears the old-fashioned way or it turns 
into equity. If government bails out a company, it is a debt that 
our children and grandchildren will have to bear. So it doesn’t re- 
duce debt in society, and this is why I have been warning against 
the stimulus packages and all of these. Transforming private debt 
into public debt is vastly more vicious than just taking the pain of 
reducing the level of debt. 

Mr. Broun. Dr. Bookstaber. 

Dr. Bookstaber. In the abstract, I don’t think that makes sense. 
In the current crisis, I think it was inevitable, because we had to 
adjust for problems that got us to where we are. So I would say 
we would want to construct a system with regulatory safeguards, 
with adequate capital, with correct incentives so that the event 
doesn’t occur where we have to move into the bailout mode that we 
had in the recent past. But my sense is that if we hadn’t taken this 
action, as distasteful and costly as it may be, the end results for 
the economy may have been far worse. 

Mr. Broun. So you believe that stimulus spending and debt accu- 
mulation and the bailouts are all necessary responses to this eco- 
nomic crisis, is what I am gathering. 

Dr. Bookstaber. Yes, I believe they were for this crisis. I don’t 
believe that as a general principle it is something that we want to 
occur, and hopefully we can take steps so that it doesn’t occur 
again. 

Mr. Broun. Dr. Taleb. 

Dr. Taleb. I don’t believe in deficit spending for the following 
reason, and it comes from the very same mathematics that I used 
to talk about tail risks. We live in a very nonlinear world — as you 
know, the butterfly effects, a butterfly in India causes a rainstorm 
in Washington. You know, these small, little — we don’t quite un- 
derstand the link between action and consequences in some areas, 
particularly monetary policy. So if you have deficit spending, it is 
debt that society has to repay someday, okay? You depend a lot 
more on expert error and projections. I showed in “The Black 
Swan,” in my book, “The Black Swan” from 27,000 economic projec- 
tions, that an astrologist would do better than economists, includ- 
ing, you know, some people here who are economists making pro- 
jections. So I don’t want to rely on expert projections to be able to 
issue a stimulus and say oh, no, no, look what will happen by 2014, 
we will be paying it back. These are more of the huge errors. 

So what is the solution? The solution is going to be that all this, 
all right, may lead to what governments have been very good at 
doing for a long time — printing, okay. And we know the con- 
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sequences of printing; everybody would like to have a little bit of 
inflation but you cannot. Because of non-linearities, it is almost im- 
possible to have the 3.1 percent inflation everybody would love to 
have. You see, a little bit of error could cause hyperinflation, or if 
you do a little less, maybe it would be ineffective. So to me, deficit 
spending, aside from the morality of transferring, you know, pri- 
vate debt into my children’s debt — okay, aside from that, because 
someone has got to buy that bond, okay, the way it may lead — you 
know, because of error in projection — [is] into printing of money. 

Mr. Broun. So from my previous questions as well as others’, I 
take it that both of you all would agree, looking in the future, not 
only with this economic crisis but in the future, to prevent other 
economic crises, the real solution is to take away the taxpayer safe- 
ty net which was implied and now with Freddie and Fannie is ex- 
press taxpayers being on the hook for this mismanagement and 
their bad decisions. Would you both agree, yes or no, that taking 
away that safety net will help people be more responsible, and we 
will have more of the sticks that my colleague was talking about 
and that they can within their own company just to protect their 
own company’s viability, et cetera, will put in place more respon- 
sible risk management and they will make better decisions. Would 
you both agree with that statement? 

Dr. Taleb. I agree with the statement, remove the safety net. 

Dr. Bookstaber. I don’t know that I can say yes or no because 
I have to envision what the future world looks like. If we make no 
changes in terms of regulation and oversight, then I wouldn’t agree 
with the notion of taking away the safety net because we have a 
flawed system where there is a notion of ‘too big to fail’ . . . where 
if certain institutions do fail, it has severe adverse consequences for 
people on Main Street. I think that we have to say, we want to get 
rid of the safety net, and to do that we need to get the corrective 
incentive structures, the correct level of oversight from regulators, 
the right capital requirements. So as an end result, that is where 
I believe we should go, but I don’t think we can be there in good 
conscience for the typical citizen without doing a better job, you 
know, in the regulatory arena. 

Dr. Taleb. I don’t understand this logic because I don’t see 
how — in 1983, when banks were bailed out, and even one of them 
was the First National Bank of Chicago. It set a bad precedent. 
Every time I heard the same argument, you hear the same argu- 
ment, “this is necessary, society can’t function, but in the future 
we’ll make sure we don’t do it again.” I don’t understand this argu- 
ment. 

Mr. Broun. Thank you, Mr. Chairman. 

Chairman Miller. Ms. Dahlkemper? Okay, Mr. Wilson? 

Mr. Broun. I think we need to go vote. 

Chairman Miller. We have been called to our votes. Thank you 
very much to this panel. We will be gone for about 20 minutes, not 
40 minutes as I earlier understood. But at that point it does make 
sense to excuse this panel, but thank you very much. It has been 
very helpful and even entertaining. And then when we come back, 
when we return we will have the second panel, although these are 
the last votes of the week so it is possible some Members will not 



84 


come back but go straight to the airport. Thank you, and we will 
be at ease. 

[Recess.] 


Panel II: 

Chairman Miller. Other Members may return or may not, but 
I think we should begin the second panel, and I also mean it when 
I say that this panel is unusually distinguished. Our witnesses are 
leading experts in their respective fields. Dr. Gregg Berman is the 
Head of Risk Business at RiskMetrics Group, which is the present- 
day descendant of the group at J.P. Morgan that created the Value- 
at-Risk methodology. He has worked with many of the world’s larg- 
est financial institutions on the development of risk models. Mr. 
James Rickards is the Senior Managing Director of the consulting 
firm Omnis Inc., is a former risk manager and investment banker 
who has been involved in the launch of several hedge funds. As 
general counsel of Long-Term Capital Management during the 
1998 crisis, he was the firm’s principal negotiator of a bailout plan 
that rescued it. And Mr. Christopher Whalen is the Managing Di- 
rector at Institutional Risk Analytics, a provider of risk manage- 
ment tools and consulting services. He volunteers as the Regional 
Director of the Professional Risk Managers International Associa- 
tion, and edits a weekly report on global financial markets. And fi- 
nally, Dr. David Colander, the Christian A. Johnson Distinguished 
Professor of Economics at Middlebury College, has written or edit- 
ed over 40 books, more than 40 books, including a top-selling Prin- 
cipals of Economics textbook and more than 150 articles on various 
aspects of economics, including the sociology of the economics pro- 
fession. 

You will also have five minutes for your oral testimony, your spo- 
ken testimony. Your written testimony will be included in the 
record for the hearing. When you have completed your spoken testi- 
mony, when all of you have, we will have rounds of questions from 
the Members who are here, which may include me repeatedly. It 
is the practice of this subcommittee, as you saw earlier, to receive 
testimony under oath. Again, I don’t think any of you have to 
worry about perjury. That would require that the prosecutor prove 
what the truth was, beyond a reasonable doubt, and that you knew 
what the truth was beyond a reasonable doubt. Do any of you have 
any objection to swearing an oath? Okay, and I think you may 
sleep easy tonight without worrying about perjury prosecution. You 
also have the right to be represented by counsel. Do any of you 
have counsel here? And all the witnesses said that they do not. If 
you would now please stand and raise your right hand. Do you 
swear to tell the truth and nothing but the truth? 

The record will show that all the witnesses did take the oath. We 
will begin with Dr. Berman. 

STATEMENT OF DR. GREGG E. BERMAN, HEAD OF RISK 
BUSINESS, RISKMETRICS GROUP 

Dr. Berman. Thank you. I would like to begin by thanking the 
Committee for this opportunity to present our thoughts on Value- 
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at-Risk and banking capital, especially in the context of the present 
financial crisis. 

My name is Gregg Berman and I am currently the Head of the 
Risk Business at RiskMetrics Group. I joined as a founding partner 
11 years ago when we first spun off from J.P. Morgan, and 
throughout that time I have had a number of roles, from leading 
research and development to leading product design, but mostly 
spending time with clients, and those clients include some of the 
world’s largest hedge funds, largest asset managers and certainly 
the world’s largest banks. During that time, and even under oath 
I feel I can say this, I have not traded any derivatives in any way, 
shape or form. 

My comments today revolve around three essential points. First, 
Value-at-Risk, or simply ‘VaR,’ was created about 15 years ago to 
address issues faced by risk managers of large, multi-asset, com- 
plex portfolios. The purpose of VaR was to answer the question: 
how much can you lose? In this context, it has actually enjoyed tre- 
mendous success, ranging from revealing the hidden risks of com- 
plex strategies to communicating with investors in a consistent and 
transparent fashion. 

Second, VaR is a framework. It is not a prescriptive set of rules. 
As such, it has been implemented in many different ways across 
a wide variety of institutions. Criticisms of VaR that focus on the 
use of normal distributions or poor historical data must be taken 
in context. These issues are often the results of specific VaR imple- 
mentations that may not have kept up with the best practices in 
the community. 

Third, most VaR methodologies utilize recent market data to es- 
timate future short-term movements in order to allow risk man- 
agers to make proactive decisions based on rapidly changing mar- 
ket conditions. This is what VaR was designed to do. Research 
shows that these estimates are indeed quite robust, but they are 
not designed to predict long-term trends, and they are not designed 
to operate when the markets themselves stop functioning. Banks, 
on the other hand, must be protected against adverse long-term 
trends and in situations where the markets actually stop func- 
tioning. This, therefore, is not the domain of Value-at-Risk. 

So how do we tackle this problem? We start by noting that the 
current crisis is driven by two primary factors: one, the failure of 
market participants and of regulators to acknowledge and prepare 
for large negative long-term trends, such as a decline in home 
prices or buildup of leveraged credit, coupled with, two, the failure 
of many institutions to accurately and completely model how these 
negative long-term trends would actually affect their financial hold- 
ings. In this context, I am using the word “model” to mean a math- 
ematical representation of a security or derivative that shows how 
its value is driven by one or more underlying market factors. Since 
both of these issues were quite well known for quite long periods 
of time, it is very hard to say that this crisis was unforeseeable, 
unknowable or a fat-tailed event. 

All market participants, including banks, must do a better job at 
modeling complex securities and in understanding how their strate- 
gies will fare under changing market conditions. For example, if 
the holders of mortgage-backed bonds would have known how sen- 
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sitive these assets were to modest changes in default rates, they 
may not have purchased them in the first place. New rules, regula- 
tions and other types of policy changes regarding better disclosure 
in data must be done in order to address this critical issue. 

But it is banks and regulators who must specifically focus on pre- 
paring more for the negative long-term trends that lie ahead and 
less on trying to predict things with probabilities. Though current 
VaR methodologies are designed to estimate short-term market 
movements under normal market conditions, regulators neverthe- 
less try to recast these models in order to measure the probability 
of long-term losses under extended market dislocations. We propose 
that it is not the model that needs to be recast, but that regulators 
need to recast the question itself. 

VaR is about making dynamic decisions, constructing portfolios, 
sizing bets and communicating risk. On the contrary, banking cap- 
ital is more like an insurance policy designed to protect against 
worst-case events and their consequences. Instead of having banks 
report probabilities of short-term losses, banks should estimate the 
losses they would expect to sustain under a set of adverse condi- 
tions chosen by regulators. The question of ‘how much can you lose’ 
is thus changed to ‘how much would you lose.’ 

The conditions that banks are tested against should depend on 
what type of events policy-makers in the public interest believe 
that banks should be able to withstand. In this fashion, models, 
probabilities, simulations and predictions are left to those making 
ongoing risk-reward business decisions, whereas the minimum lev- 
els of capital needed to ensure a bank’s survival are based on how 
regulators implement the broader requirements of policy-makers. 
Perhaps one bank needs to survive a 100-year flood whereas an or- 
derly liquidation is all that is required for a different bank. Per- 
haps all banks should be able to weather a further ten percent 
downturn in housing prices, but no bank is required to survive a 
50 percent default rate or a 40 percent unemployment rate — not 
because these events are highly improbable, but because policy- 
makers decide that this is too onerous a burden for a bank to bear. 

In summary, VaR is an excellent framework for active risk man- 
agement by banks and other financial institutions and the develop- 
ment of risk models must continue unabated. But banking capital 
serves a different purpose, and a resetting of expectations will 
allow for the development of much better solutions driven by policy 
instead of by probability. Thank you. 

[The prepared statement of Dr. Berman follows:] 

Prepared Statement of Gregg E. Berman 

I’d like to begin by thanking the Committee for this opportunity to present our 
thoughts on Value-at-Risk and banking capital in the context of the present finan- 
cial crisis. My name is Gregg Berman and I am currently the head of the risk busi- 
ness at RiskMetrics Group, a provider of risk and corporate governance services to 
the financial community. 1 have been at RiskMetrics since its founding 11 years ago 
and in the last decade have worked with many of the world’s largest financial insti- 
tutions on the development of risk models, their use by hedge funds, asset man- 
agers, and banks. 

SIMPLE ROOTS OF A COMPLEX CRISIS 

My comments today start with a rather bold assertion — the current crisis was not 
unpredictable, unforeseeable, or unknowable. In that sense I’m not sure it should 
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be classified as a fat-tailed event. Rather, it was caused by the coupling of two fun- 
damental problems, namely: 

1. the inability of market participants to acknowledge and prepare for the con- 
sequences of long-term trends, such as a protracted downward spiral in home 
prices, or a leveraging of the credit market through the use of CDS, and 

2. the inability of market participants to recognize the economic exposures they 
had to those trends through holdings such as asset-backed securities and de- 
rivative contracts. 

The fact that these issues went unchecked for many years led directly to the cre- 
ation of multiple, unsustainable market bubbles, which when burst propelled us 
downwards into a full-blown crisis. 

But if my assertion is correct and these events were foreseeable, then what does 
that imply about all the financial models and risk methodologies that were supposed 
to monitor and protect us from such a crisis? It is the answer to this question that 
I’d like to explore. 

THE INEVITABILITY OF VALUE-AT-RISK 

In the early days of risk management size was used as a primary measure of risk. 
After all, intuition tells us that $10,000,000 in Ford bonds should be ten times 
riskier than $1,000,000 in Ford bonds. But soon the market realized that 
$10,000,000 of Ford bonds is probably riskier than a similar value of government 
bonds, but not as risky as $10,000,000 of Internet start-up equity. 1 

To address these issues practitioners switched from asking “how large is your po- 
sition” to “how much can you lose.” But there is not just one answer to that question 
since for any given security differing amounts can be lost with different prob- 
abilities. One can estimate these probabilities by polling traders, by building econo- 
metric models, by relying on intuition, or by using variations of history to observe 
relevant patterns of past losses. Each of these methods has their own benefits and 
weaknesses. And unless we consider only one security at a time, it will also be nec- 
essary to make estimates of how the movements in each security are related to the 
movements of every other security in a given portfolio. 

These concepts are encapsulated by two well-known statistical terms: volatility 
and correlation. If one could measure the volatility and correlation of every security 
in a portfolio the question “how much can you lose” could be meaningfully ad- 
dressed. This process is the basis of a popular risk methodology known as Value- 
at-Risk, or VaR. 

HOW VaR IS COMPUTED AND HOW IT IS USED 

Because security valuations are often driven by underlying market factors, such 
as equity prices, spreads, interest rates, or housing prices, VaR is usually calculated 
in a two-step process that mimics this behavior. In the first step a model for the 
economic exposure of each security is created that links its value to one or more 
of underlying market factors. In the second step future trends of these underlying 
factors are simulated using volatilities, correlations, and other probabilistic meth- 
ods. These two steps are then combined to create a curve that plots potential profits- 
and-losses against the probability of occurrence. For any given curve VaR is defined 
to be the amount that can be lost at a specific level of probability. It is a way of 
describing the entire profit-and-loss curve without having to list every data point. 2 

The accuracy of any VaR number depends on how well underlying markets have 
been simulated, and bow well each security has been modeled. There unfortunately 
exists a tremendous variability in current practices and different financial institu- 
tions perform each step with varying levels of accuracy and diligence. 3 Deficiencies 


1 The matter is further complicated by derivative contracts that do not even have a well-de- 
fined measure of size. For example, what is the size of a contract that pays the holder $1,000 
for each penny-increase in the average spread throughout September between the price of nat- 
ural gas for delivery in November and the price for delivery in January? Technically the answer 
is zero since the holder owns no natural gas, but the risk is certainly not zero. 

2 Exhibit 1 on page 10 shows the potential one-day profit-and-loss distribution of selling a 
short-term at-the-money put on the S&P 500. Out of 5,000 trials we see that about 50 of them 
have losses of 250 percent or worse. Thus VaR is 250 percent with a one percent probability. 
Alternatively we can ask for the worst five out of 5,000 trials (a 0.1 percent probability) and 
observe these losses to be 400 percent or worse. 

3 The marketplace is rife with common fallacies about VaR due to poor implementations. 
When VaR first became popular in the mid-1990’s computing power limited how accurately in- 
struments, especially derivatives, could be modeled. Approximations that relied on the use of 

Continued 
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in how VaR is implemented at a particular firm should not be confused with limita- 
tions of VaR itself . 4 

But indeed there are limitations. When computed according to current best prac- 
tices, VaR is most applicable for estimating short-term market volatilities under 
“normal” market conditions. These techniques are based on over a decade of well- 
tested research demonstrating that in most circumstances recent market movements 
are indeed a good predictor of future short-term volatility. VaR models have seen 
tremendous success in a wide range of applications including portfolio construction, 
multi-asset-class aggregation, revealing unexpected bets, investor communication, 
the extension of margin, and general transparency. 

As such, VaR has become an essential part of risk management, and when prop- 
erly integrated into an overall investment process it provides an excellent frame- 
work for deploying capital in areas that properly balance risk and reward. 

VAR AND BANKING CAPITAL 

So why did this not foretell the current crisis? First and foremost, many institu- 
tions and market participants did not perform step one correctly — they failed to cor- 
rectly model how their securities would behave under changing market conditions. 
This failure is one of the leading causes of current crisis . 5 

The second issue is where banking capital comes in. Recall that our crisis stems 
from long-term trends, not short-term volatility. And as mentioned, most of today’s 
VaR techniques are only applicable for estimating potential short-term movements 
in well-functioning markets . 6 But it is long-term trends and non-functioning mar- 
kets that are the concerns of banking capital. 

Nevertheless regulators rely on VaR as the basis for many bank capital calcula- 
tions . 7 And even today they continue to recast VaR-like models in order to address 


so-called normal distributions (bell-shaped curves) were often required. Also, the amount of mar- 
ket data that could be used, and the frequency at which this data was updated, was limited 
by technical and mathematical challenges resulting in further approximations. However, by the 
early part of this decade many of these challenges were overcome and today’s simulation tech- 
niques do not rely on normal distributions and are not restricted by limited data. Unfortunately 
many institutions with older implementations still use somewhat outdated and approximate 
methods that do a poor job in estimating the risk of multi-asset, derivative-heavy portfolios. 

4 One fundamental criticism of VaR is that it can be “gamed” or manipulated since one num- 
ber cannot by itself represent or reveal all possible “tail-loss” events. This is easily rectified by 
simply asking for VaR numbers at more than one level of probability, by computing the average 
of all losses comprising a tail event (often called conditional VaR or expected loss), or by exam- 
ining the entire distribution of estimated future losses and their corresponding probabilities. 

5 Many institutions and market participants did not recognize nor understand how their port- 
folios and strategies would be affected by a fall in housing prices or a widening of credit spreads. 
Regulators had even less information on these effects and almost no information on how they 
were linked across institutions. 

It could be argued that if investors had understood the nature of the mortgage-backed products 
they had purchased, many would not have purchased them in the first place (which would have 
significantly curtailed the formation of the bubble itself). If regulators had understood how CDS 
contracts inherently lever the credit markets they may not have allowed their unbridled expan- 
sion. And if insurance companies understood that changes in the mark-to-market values of their 
derivative contracts would require the posting of collateral to their counter-parties many would 
not have entered into those deals. None of these decisions involve predicting the future or mod- 
eling fat tails. They do involve understanding the present, spending time on the details of finan- 
cial instruments, and being incented to care about their risk. 

Tackling these significant shortcomings may require new regulations regarding data avail- 
ability, disclosure, and the analytical capabilities of each market participant. Central oversight 
of the markets themselves will be needed to monitor, and sometimes even limit, actions that 
could trigger systemic risk and future liquidity crises. 

6 There is nothing endemic to VaR that limits its applicability to short-term estimates or func- 
tioning markets. However, current methodologies are optimized for those conditions and this is 
where most parameters have been tested for proper use. Research into new models that length- 
en the prediction horizon and include factors like liquidity to account for non-functioning mar- 
kets is underway. As development of these methodologies progresses we may see the domain 
of VaR extended into more areas of risk. 

7 One technique employed to “fix” the short-term aspect of VaR models is to utilize long-term 
historical data as the basis for “better” future estimates. This is a very common but dangerous 
practice since it both invalidates any estimates of short-term volatility (preventing proper use 
by risk managers trying to be reactive to rapid changes to the market) and it doesn’t actually 
provide any better estimates of long-term trends. For a complete discussion on this and other 
related topics see included reference by Christopher Finger ( RiskMetrics Research Monthly — 
April 2009) and references therein (including a March 2008 report issued by the Senior Super- 
visors Group on their study of how risk was implemented at a variety of large banks). 
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VaR’s perceived shortcomings. 8 We propose that it is not the model that needs to 
be recast but rather the question that regulators want the model to address. 

POLICY-BASED BANKING CAPITAL 

We believe that the foundation of banking capital is rooted in the following two 
questions: 

1) What are the adverse events that consumers, banks, and the financial sys- 
tem as a whole, need to be protected against? 

2) What is required from our banks when those events occur? 

This is not the domain of VaR. On the contrary, banking capital is more like an 
insurance policy designed to protect against worst-case events and their con- 
sequences. Instead of having banks report probabilities of short-term losses, banks 
should estimate the losses they would expect to sustain under a set of adverse con- 
ditions chosen by regulators. The question of “how much can you lose” is thus 
changed to “how much would you lose.” 

The conditions that banks are tested against should depend on what types of 
events policy-makers decide that, in the public interest, banks should be able to 
withstand. In this fashion models, probabilities, simulations, and predictions are left 
to those making ongoing risk-reward business decisions whereas the minimum lev- 
els of capital needed to ensure a bank’s survival are based on how regulators imple- 
ment the broader requirements of policy-makers. Perhaps one bank needs to survive 
a hundred-year flood whereas an orderly liquidation is all that is required for a dif- 
ferent bank. Perhaps all banks should be able to weather a further 10 percent down- 
turn in housing prices, but no bank is required to survive a 50 percent default rate 
or a 40 percent unemployment rate — not because these are highly improbable, but 
because policy-makers decide that this is too onerous a burden to expect a bank to 
bear. 9 

To summarize, we believe that key differences between the needs of risk manage- 
ment and banking capital suggest different solutions are required. And in doing so 
each field can separately develop to meet the ever-expanding array of challenges we 
face today. 


8 See included reference by Christopher Finger ( Risk-Metrics Research Monthly — February 
2009) containing our comments on the Basel committee’s proposed Incremental Risk Charge — 
an extension that uses VaR for additional types of capital charges. 

9 The recent stress-tests conducted on banks by the Federal Reserve is an excellent example 
of how policy, as opposed to probability, can help set capital requirements. This should not di- 
minish the role of simulations and the use of models to explore possibilities and uncover unex- 
pected relationships, but this should be a guide of what the future may bring, not a prediction 
of what it will (or will not) bring. 
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Exhibit 1: VaR and the Profit and Loss Curve 


Sample Position = S&P 500 Put Option 
5000 Simulations of Potential Profit and Loss 
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Amid the financial crisis of the past year, one of the 
many alleged villains has been the Basel II banking 
capital standard. Lost in the debate is the not so sub- 
tle point that the standard is still evolving, and in fact 
the Basel Committee is taking steps to address many 
of the regulation's shortcomings. One such step is 
the recent publication of the third consultative docu- 
ment on assessing incremental capital in the trading 
book. Though not finalized, this new regulation rep- 
resents an important step to closing a gap in the exist- 
ing capital rules— credit risk in trading portfolios- 
that has been severely exposed by recent events. 

We comment here on the new proposal, from both 
a practical and fundamental perspective. To do so. 
wc begin with a statement of the Committee’s goals: 
some of which have been explicitly stated, some 
which we can infer from prior documents and some 
which we would assert should be goals, even if the 
Committee is not in a position to stale them out- 
right. With a set of broad principles defined, we 
then proceed to question whether the specific pro- 
posal achieves the goals, and to make a few modest 
recommendations as to how they may be better met. 

A bit of history 

If the one overarching theme of the first Basel Ac- 
cord was consistency, that of the second accord 
©2009 RiskMetrics Group. Inc. All Rights Reserved. 


must be risk sensitivity, along with the acknowledge- 
ment that banks themselves are capable of measuring 
risks. In fact, Basel II was not the first appearance of 
risk sensitivity in the capital guidelines: under the 
1998 Market Risk Amendment (MRA) to the orig- 
inal Basel Accord, banks were given the option to 
base part of their minimum capital requirement on 
their own assessment of the risk of their trading port- 
folio, subject to certain restrictions and standards. 

For the most part. Basel II was notable for extend- 
ing the internal models idea to banking portfolios. 
The trading book capital treatment, though updated 
somewhat, did nol change dramatically from how it 
was set forth in the MRA, under which banks are 
assessed minimum capital in two parts: general and 
specific. 

The general risk charge is intended to cover risks due 
primarily to systemic market moves; hanks are per- 
mitted to use their own models to measure this risk, 
and capital is defined as the VsIue-at-Risk (at 99% 
confidence over a ten-day risk horizon) multiplied by 
a scaling factor, which is subject to supervisory dis- 
cretion. but is no less than three. The specific nsk 
charge is to cover the impact of security- or issuer- 
specific events: spread widening, credit migration, 
default, and so on. As in 1998. many risk models 
did not account for such events, the original specific 
risk charge was a standard supervisory formula. As 
tune went on. more sophisticated banks were able to 
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secure approval lo expand their internal model cov- 
erage to specific risk. 

While Basel II left the general and specific capital 
charges largely unchanged, it recognized the need to 
better cover the default risks inherent in trading port- 
folios. and bid out a number of high level principles 
for treating incremental default risk. As the princi- 
ples were presented in 2005. they were not (yet) a 
reaction to significant trading book losses, but rather 
a recognition of the amount traded credit risk had 
grown since the MRA. 

In October 2007. the Basel Committee published a 
consultative document defining an Incremental De- 
fault Risk Charge. In response to both industry 
comments and market developments, the Commit- 
tee expanded the charge to include migration risks 
(and consequently renamed it the Incremental Risk 
Charge, or IRC). The third consultative document, 
published in January 2009, refines the definition of 
the IRC. and formalizes the total trading book capi- 
tal requirement as the sum of the general and specific 
charges from the MRA and the new IRC. 

The IRC itself is defined as the VaR of the unsecuri- 
lized credit products in the trading book, at a 99.9% 
confidence level over a one-year horizon. Recogniz- 
ing that imposing a one-year holding period is un- 
realistic for many trading positions, the IRC permits 
a liquidity horizon for each instrument, with the no- 
tion that at this horizon, an instrument may be rebal- 
anced in order to maintain a constant level of risk. 
Other than the restriction that no instrument have a 
liquidity horizon shorter than three months, the defi- 
nition of the model and parameters for the IRC is left 
largely to the banks. 


A set of principles 

Though tempting to jump straight lo the specifics, it 
is important to first establish the motivations for the 
IRC specifically and model-based capital generally. 
We should begin with the very principle of risk sen- 
sitivity: required bank capital should depend on the 
risks of the bank. Moreover, it is probably inevitable 
that a risk sensitive capital regime will depend on 
bank internal models. Taken together, these notions 
are under fire, with the perceptions that through this 
crisis, banks were undercapitalized (a fair point) and 
incapable of modeling their own risks (an overreac- 
tion). These perceptions have prompted cries for a 
“back to basics" approach, but this is dangerous. 

If by “back to basics” we mean a return to the sim- 
ple capital requirements of the original Basel Ac- 
cord. we should recall what motivated Basel 11 in the 
first place. The criticism of the original accord was 
that it was basic and inflexible, and as such produced 
an incentive for banks to remove low yielding (but 
safe) credits from their balance sheet, while retaining 
higher yielding (but riskier) credits. Such an incen- 
tive was no more than an academic point in a world 
of buy-and-hold credit, but with the rise of collat- 
eralized debt obligations (CDOs) and credit default 
swaps not long after, banks had a ready mechanism 
to act on those incentives. It is fair to say that the 
original development of CDOs, the descendants of 
which wound up at the heart of this crisis, owes to the 
skewed incentives of the old. basic capital regime. 
That is not something we should aim to go back to. 

If, on the other hand, by "back to basics”, we mean a 
turn to more transparent capital requirements, that is 
a principle to hold to. 
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So while we should agree that a transparent, nsk sen- 
sitive capital regime is still desirable, it is impossible 
to neglect the point that banks were undercapitalized. 
So the next basic principle is that we should find 
ways to increase capital standards, focusing in partic- 
ular on the trading book, where the significant losses 
have occurred. At the same lime, it is important to be 
mindful of the rules for banking book capital, as we 
do not want the capital standards to produce skewed 
incentives for where particular positions are placed. 

An issue that has lurked in the background, but has 
come to the front in the IRC discussion is the tension 
between the regulatory risk horizon and the hori- 
zon at which banks manage their trading portfolios. 
From the regulatory point of view, the concern is the 
bank as a “going concern". The aim of minimum 
regulatory capital, then, is that banks can withstand 
losses, and are able to continue their normal activi- 
ties. without the need to raise additional capital. The 
dispute between industry anti the regulators over the 
appropriate regulatory risk horizon, in fact, has been 
essentially a proxy battle over the frequency at which 
banks are able to raise capital. Whereas back in 
2007. early industry responses to the IRC proposals 
argued for a shorter risk horizon, asserting that banks 
could raise additional capital if they suffered capital 
depleting losses, the events of the last year has ren- 
dered such arguments void, and it is the regulators 
with the upper hand in pushing for a longer horizon. 

With their going concern, long horizon view, the reg- 
ulators have put themselves firmly in the camp of 
seeking models for banking (or trading) practices. 
Regulatory capital, then, is intended to support not 
just possible losses on existing positions, but on all 
of the positions throughout the next year, as a bank 


is assumed to continue its normal operations. 

Internal trading risk management, on the other hand, 
focuses on positions. As such, the risk horizon is not 
the frequency over which the bank can raise capital, 
but rather the frequency at which positions can be 
traded. This horizon, particularly with trading port- 
folios and even accounting for stressed levels of liq- 
uidity. is markedly shorter than the capital horizon. 
It is this problem — short horizon risk on constant 
portfolios — that market nsk managers have focused 
on for over a decade. And even if the models may be 
imperfect or incomplete, their short honzons at least 
allow for empirical validation. 

The MRA rules represent a compromise between 
the regulatory and trading horizons. The regulations 
recognize that banks are capable of modeling short 
honzon nsks. yet also dial the purpose of capital is 
more than absorbing 99% worst case, ten-day losses. 
Thus, banks are permitted to estimate their short 
horizon VaR. and the regulator applies two mecha- 
nisms to convert this measure, at least qualitatively, 
into a long horizon one. 

The first mechanism is that, while the bank calcu- 
lates iLs VaR daily, it is assessed capital on a sixty- 
day moving average of the VaR measure; this adjust- 
ment is an admission that while the short term risk of 
a trad mg portfolio may indeed be quite volatile, the 
capital required to support longer term trading ac- 
tivities should be smoother. 'Hie second mechanism 
is the multiplier the short horizon VaR is scaled by 
a regulatory factor, which is typically between three 
and five. The MRA rules, then, display a principle 
of compromise, and are a balance between quantita- 
tive risk sensitivity, as measured by the banks them- 
selves. and qualitative prudential standards, as im- 
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posed by the regulator. 

A consequence of this balance is that capital is not 
defined in precise statistical terms. Basel defines nei- 
ther the horizon nor the confidence level they con- 
sider prudent, but enforce prudence through their 
subjective choices of multipliers. The new rules till 
this balance, attempting to define capital in a more 
rigorous statistical framework. And they impose on 
banks the task of modeling not just their positions, 
but their practices, over a hon/on long enough to suit 
the regulator. 

Finally, some models, as the saying goes, are useful. 
But with a capital regime encouraging new model de- 
velopments. it is not just models that should be use- 
ful. We should also require dial discussions about 
models be useful, and that the byproducts of these 
discussions, be they other models or simply height- 
ened awareness of other issues, be useful as well. 

Recipe for trouble 

In the industry responses to the prior IRC consulta- 
tive documents, a consistent complaint has been the 
lack of an overarching conceptual framework to the 
new rules. The confusion stems foremost from the 
definition of capital as the sum of the old MRA cap- 
ital. based on a ten-day 99% VaR. and the new IRC 
capital, based on a one-year 99.9% VaR and cover- 
ing a slightly different set of sources of risk. As with 
the MRA. there is still no precise statistical defini- 
tion of capital, but the new rules are more frustrating 
in that they tease with the illusion of precision (all 
of the components are defined statistically), and yet 
the final aggregated number is no more than a recipe. 


with no conceptual framework to justify it. What in 
the end does it really mean to take the sum of ten-day 
99% and one-year 99.9% VaR? 

While only the most ardent modelers (still) believe 
in precise estimates of such things as a 99.9% one- 
year VaR. it is not just these zealots crying for a con- 
ceptual basis for capital, the better to apply yet more 
complex techniques. Even the more rational among 
us should recognize that the lack of a conceptual 
framework is dangerous, and not just aesthetically 
bothersome. When we have no conceptual frame- 
work, capital in the end is a recipe. When a recipe is 
simple, such as multiplying VaR by three, it is easy to 
anticipate how the recipe will behave in new circum- 
stances. When a recipe is more complicated, such as 
the sum of two inconsistently defined VaR figures, 
we do not know what we can really expect. 

The examples in the paper by Breuer et al (2007) 
should serve as warning. Just adding the risk mea- 
sures arising from different sorts of risks does not. as 
popular wisdom would argue, necessarily produce a 
conservative answer. Too many of us have gotten too 
accustomed to the behavior of the simplest of mod- 
els, where risks add in simple ways: so we propose 
a recipe that under the simplest of models is guaran- 
teed to behave as we desire, but under realistic mod- 
els. we have no idea what will happen. 

Industry is not blameless here either. One of the 
early industry counterproposals to the IRC was to 
add these distinct risks via the standard formula for 
adding standard deviations, with an “effective corre- 
lation” calibrated to match actual model results in se- 
lect cases. This proposal was no better than Basel’s, 
in that it replaced one recipe for adding risks with no 
conceptual basis with another more complicated, but 
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equally unfounded recipe. Basel, while not coming 
up wiih something richer, was wise at least to discard 
this industry proposal. 

What what else to do? 

A better industry proposal sought to bring some con- 
sistency to the standard market risk and IRC charges. 
This proposal was based on the observation that three 
times the ten-day 99% VaR is close to the VaR for 60 
days at 99.9% confidence, assuming a Gaussian dis- 
tribution and (lie standard lime scaling. Sixty days 
being a plausible time horizon for overall trading 
book risk, and 99.9% being consistent with the sol- 
vency standard initially proposed by Basel, the in- 
dustry proposed using a 60-day, 99.9% VaR for all 
trading book capital. The unportanl consequence of 
this is that it allows for a single model, encompassing 
all nsk sources, rather than the arbitrary division of 
risks of the IRC, and the resulting specter of falsely 
conservative capital rules. 

The latest proposal from Basel sticks to the separate 
capital charges and risk horizons, an apparent blow 
to the industry proposal. One rationale for the per- 
sistent one-year horizon is that Basel wishes to avoid 
too large a discrepancy between the capital for an 
asset in the banking book (which is based on a one- 
year horizon) and that for the same asset in the trad- 
ing book (the IRC). A second rationale is clearly that 
Basel desires to raise capital levels by more than the 
sixty-day horizon would imply. 

With the discussions focusing on the risk horizon and 
solvency standard, it is disappointing that the choice 
of risk statistic has never been seriously reconsid- 


ered. In the consultative document, the Committee 
makes it clear that a shortcoming of the MRA capital 
rules is that they are insensitive to the sorts of rare 
but large events that the IRC is intended to address. 
Iheir reaction to this is to push out both the horizon 
and the confidence level, seemingly raising both the 
overall capital and the risk sensitivity of the rule. 

This approach is misguided. A measure such 
as VaR — a specific quantile of a probability 
distribution — is insufficient to distinguish the risk of 
portfolios with differently shaped distributions. In 
a 1998. MRA era view, where all distributions are 
Gaussian (or close) the VaR is really no more than a 
scaled version of the standard deviation. And if all 
portfolios are distributed with the same shape, this is 
enough to differentiate more from less risky portfo- 
lios. In this setting. VaR was a sensible choice on 
which to base a risk sensitive capital regime. 

But we are in a different setting now. If our focus is 
on rare but large events, then the new capital regime 
should be able to differentiate between two portfo- 
lios that differ only in that one is exposed to a rare 
but catastrophic event. As long as the likelihood of 
the rare event is significantly less than the VaR con- 
fidence level, the VaR of these portfolios is almost 
equal. One remedy is to make our confidence level 
stringent enough to cover what we think will be the 
likelihood of such rare events. But this is a fragile 
regime, and only risk sensitive in the particular range 
of probabilities that are defined a priori. A different 
statistic — in particular expected shortfall, or the ex- 
pected loss, conditional on experiencing a loss in the 
tail — produces a measure that is at once sensitive to 
the risks we desire to measure, and not as sensitive 
to the arbitrary choice of confidence level. 
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Constant level of frustration 

By far the most controversial element of the Basel 
proposals has been the notion of an assumed coastanl 
level of risk. The constant level of risk and its com- 
panion notion, the liquidity horizon, are in a sease 
concessions by the Committee that a one-year risk 
horizon with fixed positions is unrealistic for most 
trading portfolios. This author has been tom on the 
proposal, in fact praising it in an earlier column . 1 

To rationalize somewhat, our praise at the time was 
for the constant level of risk notion as the first ex- 
plicit acknowledgement that with long ask horizoas. 
it is imperative to address the problem of portfolio 
aging. The assumption of constant positions, while 
adequate for short trading horizons, is unrealistic in 
the context of the “going concern" outlook that Basel 
aims to apply for a'gulalory capital. We maintain 
that Basel is due praise for promoting what is the first 
serious industry discussion of this topic. 

Praise is due as well for the emphasis on the liquid- 
ity horizon. While its role in this particular modeling 
framework is debatable, the assessment of a liquidity 
horizon for trading positions is a worthy goal in its 
own right . 2 That the liquidity horizon for trading po- 
sitions is a quantity that can materially impact capital 
requirements is a guarantee that these will be the sub- 
ject of significant scrutiny and debate. Regardless of 
the ultimate application of the liquidity hon/on. this 
debate will necessarily raise awareness and promote 
transparency of liquidity risks. While not a slated 
goal of the IRC proposal, this transparency of liquid- 
ity is a desirable side benefit, and discussioas thereof 


most definitely useful. 

The constant level of risk framework, on the other 
hand, is an invitation to arbitrary recipes, and for the 
more cynical, to manipulation of the minimum capi- 
tal through unneeded complexity. In a utopian world, 
we would build a risk model that simulated each as- 
set through time. For each asset, at the appropriate 
liquidity horizon, we would sell the asset, realizing a 
gain or loss, and reinvest the proceeds. The reinvest- 
ment would be into a new hypothetical asset with the 
same characterisucs as the original position at initi- 
ation. The motivation for such a rebalancing strat- 
egy is that it is expected to reduce risk relative to the 
strategy of holding all positions for an entire year. 

But the effects of the rebalancing strategy are subtle, 
and it is unclear when the strategy actually does re- 
duce risk. Among other things, (he outcome depends 
on three features of the utopian simulation: 

• Structure of credit (or risk) curses. It is a com- 
mon that an issuers' credit curve is upward 
sloping, that ls. that if a default event is to oc- 
cur. it is more likely to occur in the long term 
than in the near future. If this is true for most 
issuers, then rebalancing reduces our risk by 
exposing us to a succession of “near futures" 
rather than a single long term. 

• Inter-period correlations. In efficient markets, 
we can typically assume (and validate) that 
market moves in one period are uncorrelated 
with market moves in the next. But with rat- 
ings migrations and defaults, it is much more 
difficult to argue market efficiency, and ef- 


'Scc Finger (2008). 

•’Indeed, in a parallel consultative process. Basel has issued a set of principles for sound liquidity risk management. 
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feels such as ratings momentum are well doc- 
umented. 

• Risk dynamics. It is acknowledged that both 
default probabilities and volatility vary with 
tune. Thus, it is not cerlain that the risk of a 
BBB position in six months, after my initial 
BBB position has downgraded, will in fact be 
a constant level of risk. 

In liquid markets, where risk factors evolve contin- 
uously (or approximately so), it is plausible that we 
might defend such a set of assumptions. But ratmgs 
migrations and defaults do not evolve continuously, 
and what is more, are subject to very human, dis- 
crete processes: when docs a linn choose to file for 
bankruptcy, or an analyst to review a rating. When 
migrations and default are the risk factors, clarifying 
a model with rebalancing will always come down to 
some articles of faith. And when capital depends in 
complex ways on articles of faith, it is naive not to 
believe that faith could well be corrupted. 

It is naive as well to believe practical a utopian sim- 
ulation with asset-by-asset rebalancing; any imple- 
mentation of an IRC model will be a mere approxi- 
mation to this ideal. At least in the special case where 
there is a single liquidity horizon, the consensus 
emerging in the industry is to obtain the one-year dis- 
tribution by summing successive short-horizon port- 
folio losses: for a liquidity horizon of three months, 
we define the three month portfolio loss distribu- 
tion. then sample four limes from this. Embedded 
in this operation is the assumption that in our suc- 
cession of "near futures”, each period is independent 
of the next. Mathematically, this procedure amounts 
to computing a convolution of the short horizon loss 
distribution. 


The convolution approach, while tractable and seem- 
ingly consistent with the spirit of the IRC definitions, 
is problematic. Most importantly, the approach does 
not define a model in terms of a process; we have 
not specified how the portfolio evolves from one pe- 
riod to the next. As such, there is no natural way to 
generalize the approach to a portfolio with a mix of 
liquidity horizons, a situation sure to arise in prac- 
tice. Consider a portfolio composed of three-month 
and six-month liquidity horizon positions. Applying 
the convolution to either portion of (he portfolio is 
s tra ighl forward, but how do we make sense of the 
dependence between the six month loss experience 
on the less liquid subportfolio and the month four to 
month six loss experience (including hypothetical re- 
balanced positions) of the more liquid portfolio? 

In the end, the convolution approach, which appears 
to Ik* the only tractable way to address die constant 
level of risk proposal, is in truth no more than a scal- 
ing rule, albeit an opaque and complicated one. 

A humble proposal 

Acknowledging the basic principles and goals of the 
IRC proposal, we offer a recommendation for a slight 
modification. We have in a sense a cap on what the 
IRC should be, in the form of the Basel II internal 
ratings-based (IRB) capital charge— the capital that 
would be required were the assets in question to sit 
in the banking book. In the limit where our trading 
positions are highly illiquid, these positions should 
be treated exactly as banking book positions. 

We should establish a limit on the other end of the 
spectrum: what the IRC should be for very liquid 
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positions. Basel has established that they view three 
months as a minimal liquidity horizon, and so we 
should detine this liquid- IRC based on a three-month 
holding period. While it is appealing to establish 
a risk measure (VaR) and confidence level (99.9%) 
to be consistent with the IRB charge, we are faced 
with the shortcomings of the VaR measure discussed 
previously. It would be sensible, then, to base the 
definition on an expected shortfall, with confidence 
level set to match the prudential standard of the IRB 
charge, at least in some special cases. The Commit- 
tee should reserve the right to apply a capital mul- 
tiplier in this case, in order to raise overall capital 
levels without compromising the desirable risk sen- 
sitivity of tile measure. 

With our two limits established, it remains to specify 
how capital should be computed for portfolios of an 
arbitrary liquidity horizon. Here, we must admit that 
the constant level of risk approach, while appealing 
in spirit, is unattractive in practice, and will in all 
likelihood lead to obfuscation without any real mod- 
eling advances. The liquidity horizon, on the other 
hand, is a worthwhile endeavor on its own. We pro- 
pose. then, that banks establish and defend liquid- 
ity horizons for all positions, subject to a prudential 
regulatory floor, and calculate the average liquidity 
horizon across the portfolio. Moreover, the liquidity 
horizon definition should play a role in the needed 
definition of “illiquid positions" within the specific 
IRC rules. 

The final IRC charge should then be obtained by 
a simple interpolation rule (not necessarily linear, 
but still transparent) between the two limiting cases. 


based on the portfolio average liquidity horizon. 
Granted this proposal is itself but a recipe, and not 
a coherent statistical definition of solvency. But in 
the spirit of the old MRA compromise, it is a recipe 
that makes fully transparent the dependence of cap- 
ital on the liquidity horizon, while at the same time 
preserving an appropriate level of capital for the most 
and least liquid portfolios. Most importantly, it is 
a recipe that encourages the industry to focus its 
research efforts on items dial are valuable in their 
own right — short-term default probabilities, liquidity 
horizons, basis risks— and limits the most onerous 
operational requirements and least valuable model- 
ing discussions at the same lime. 

Further reading 

• Basel Conunittee on Banking Supervision 
(2008). Principles for sound liquidity risk 
management and supervision. September. 

• Basel Conunittee on Banking Supervision 
(2009). Guidelines for computing capital for 
incremental risk in the trading book. Consul- 
tative document. January. 

• Breuer, T. et al (2007). Regulatory capital for 
market and credit risk interaction: Is current 
regulation always conservative? Conference 
on the Interaction of Market and Credit Risk, 
Berlin, December. 

• Finger, C. (2008). The once holy grail. Risk- 
Metrics Research Monthly. January. 
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In February, wc commented on a Basel Committee 
proposal to implement a capital charge (the Incre- 
mental Risk Charge, or IRC) to cover default and 
migration losses in the trading book. Last month, 
the Financial Services Authority (FSA) of the UK 
released the Turner Review, an analysis of the ori- 
gins of our financial crisis and recommendations for 
the future of financial services regulation, along with 
a discussion paper staling a more oflicial FSA policy 
stance. Unlike the detailed proposal for the IRC. the 
FSA documents represent a broader review of regu- 
lation and capital generally. 

Neither FSA document is guarded in its assessment 
of financial supervisors in the runup to the crisis. The 
discussion paper summarizes: 

...prudential regulation was loo weak 
at a micro-prudential level, and al- 
most completely lacking at a macro- 
prudential level. (Paragraph 4.2) 

Central to the calls for macro-prudential regulation 
is the reaffirmation of the supervisors' role in ensur- 
ing the stability of the financial system. The papers 
propose that regulators take an active role in identi- 
fying systemic risks - easing of credit standards, ris- 
ing systemic leverage, falling risk premia through 
both macroeconomic analysis anti knowledge of the 
actions of specific institutions. This is an ambitious 
step, broadening the mandate for regulation gener- 
<©2009 RiskMctrics Group. Inc. All Rights Reserved. 


ally, but also abutting cooperation between the dif- 
ferent agencies that today perform macroeconomic 
and institution specific analyses. 

For the micro-prudential level — the supervision of 
individual institutions — the strongest call is for a 
"fundamental review" of the existing minimum cap- 
ital rules. The tone of the papers make clear that the 
FSA holds strong views on where this review should 
lake us. As for Value-at-Risk (VaR) models, the FSA 
slates that the bunion of proof lies with the industry 
to show that VaR is appropriate for capital rules. 

But such a challenge sets up the review as a (po- 
tentially one-sided) discussion focused on the merits 
(or demerits) of VaR. This is far less productive than 
starting with a review of what trading book capital 
should be. with the notion that if a short-term mea- 
sure of risk is appropriate, then VaR should prove 
Us place. Our ami in this note, then, is to address 
soiir* of the wrong reasons the FSA has put forth 
to cast VaR aside, offer some good reasons of our 
own, and comment on where short-term risk mea- 
sures could contribute in a future trading book capital 
framework. 

What went wrong? 

Both FSA documents begin with the same chapter 
entitled “What went wrong'?" The chapter reviews 
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(he macroeconomic environment and the growth of 
the securitized credit model, and then moves to the 
now obligatory section on ’’Misplaced reliance on so- 
phisticated maths”. Central to this section is an ex- 
hibit (Box I A in the Turner Review) summarizing in 
three main points the FSA views on the deficiencies 
of VaR-based estimates of risk. 

One of the three VaR deficiencies is the "failure to 
capture fat-tail risks", which is further elaborated: 

Short-term observation periods plus as- 
sumption of normal distribution can 
lead to large underestimation of prob- 
ability of extreme loss events. 

While the symptom here (extreme loss events were 
underestimated) is uncontroversial. both aspecLs of 
the diagnosis are misguided. 

To be clear. VaR models are intended to forecast (in 
a statistical sense) the possible loss on today's port- 
folio. based on today's market, over a short (one day 
to one month, for instance) risk horizon. Almost al- 
ways. these models rely on historical price moves 
and some statistical inference to perform forecast- 
ing. In regulatory language. VaR by definition is a 
point-in-time measure. 

One implication is that VaR models are verifiable: 
we can track both our forecasts of possible loss and 
the actual loss experience over time, and validate 
whether these two have the statistical relationship 
that they should. Banks perform this backtesting as 
part of their regulatory and disclosure requirements. 
Among the standard metrics is the number of VaR 
exceptions, that is. the number of days on which ac- 
tual trading losses exceeded the VaR forecast. 


Campbell (2009) recently surveyed bank VaR disclo- 
sure for 2008. Even in such an eventful year, there 
was a wide range of model performance. One of the 
better performers. Bank of America, reported two 
VaR exceptions at 99% confidence, almost precisely 
what one would expect. This was an improvement 
on the bank model's performance in 2007, which the 
bank attributed to their move to more frequent data 
updates in an effort to react more rapidly to higher 
volatility. In fact, this move to more reactive volatil- 
ity measures was one of the best practices cited by 
the Senior Supervisors Group (SSG) in March 2008. 
This example also shows that good backtesting re- 
sults do not guarantee that a bank had a peaceful year. 

At the other end of the spectrum was UBS. which 
recorded 50 exceptions at 99% confidence. In other 
words. UBS observed what they thought was a one- 
in- a- hundred event on average once per week, a spec- 
tacular failure in risk forecasting. One of the bank's 
comments on this poor performance was to assert 
that then results "highlight the limitations of VaR”. 
Such a broad dismissal of risk forecast ing based is 
especially feeble in light of other banks’ success. 

A closer look at the UBS disclosure is illuminat- 
ing. In its annual report, the bank discloses that it 
utilizes five years of historical returns, with equal 
weighting, to produce its VaR forecasts. It warns that 
this method "does not respond quickly to periods of 
heightened volatility”. Indeed. 

Thus, consistent with the SSG's recommendations, 
but at odds with the FSA evaluation, it was long ob- 
servation periods that led to underest unation of risks. 
In the short term, nsk changes, and risk models must 
react. The use of short 1 observation periods for VaR 


‘Or effectively short, in the case that past data is weighted unequally 
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forecasting is necessary to make good forecasts, and 
not just a convenient choice that lets us save money 
on data storage. 

False relics 

This brings us to the second part of the diagnosis 
the use of the infamous normal distribution— which 
has been fodder for a thousand popular press vilifica- 
tions of risk models. While it is tme that the normal 
distribution is an oversimplification of empirical loss 
experience, the continued burning in effigy of Karl 
Friedrich Gauss has distracted the dialogue from nu- 
merous other points. And while we can ignore the 
counterproductive dialogue in the tabloid press or 
chat forums, the regulatory dialogue is more impor- 
tant. Unfortunately, to be heard above the din. a bit 
of yelling is in order. 

First, any discussion of probability distribution (the 
description of our ignorance) must coine only after 
using the available information to forecast what we 
can. in this case volatility. If asked to propose a dis- 
tribution of the heights of a group of school children, 
it would be absurd to start deriving tail exponents be- 
fore asking how old the children were. 

Second, criticisms of the normal distribution are not 
new, and there exist numerous VaR model imple- 
mentations that use alternatives. Not all of these are 
improvements, however. Those that use non-normal 
distributions but do not forecast changes in volatil- 
ity perform categorically worse than those that stick 
with the beleaguered normal distribution but react 
appropriately to changing market conditions. 

Third, the search for the right fat-tailed distribution 


at best gives us a more accurate view of the static part 
of our risks. As such, it is in some ways equivalent to 
shrugging our shoulders and saying "Shit happens". 
Maybe we recognize that market moves of ten times 
our volatility forecast are significantly more likely 
than we thought, not one-in-a-billion events but one- 
in-a-lhousand events. But ill is teaches us nothing 
about the dynamics of the market we are trading, and 
gives us no warning signals of when those events are 
becoming even more plausible. The plea for the right 
fat-tailed distribution is the medieval response to Ihe 
Plague, not to understand hygiene, germs or conta- 
gion. but to blame misfortune on the unknowable and 
immeasurable. 'litis reaction is dangerous, as it pre- 
supposes not only that we don’t know, but that we 
can’t know. Our only hope is to trust in someone 
endowed with divine knowledge of Ihe unknowably 
improbable. 

Bigger distractions 

The most dangerous distraction then is from an En- 
lightenment response to the Plague— a real analysis 
of what could have made underest rotations of risk 
better, not just by making them larger, but by making 
them more timely. There are two crucial areas which 
have been all loo often ignored. To their credit. Ihe 
FSA does focus on one of these. 

Ihe second of the VaR deficiencies Is "Failure to cap- 
ture systemic risk”, in which the FSA cites the as- 
sumption that "each institution is an individual agent 
whose actions do not themselves affect Ihe market", 
and asserts that “interconnected market events can 
produce self-reinforcing cycles which models do not 
capture". This is potentially a much more productive 
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discussion than (hat of fai-lailed distributions, as it 
leads us away from a view purely based on historical 
returns data, and holds the promise of more than just 
larger risk estimates overall. Kit more timely signals 
of when risk has increased. 

In fact, the language here lets us resolve one of the 
damaging rhetorical corners that the risk community 
has painted itself into. It is common to state that VaR 
models work in “normal markets", but without ever 
delining what "normal markets" are. The lack of a 
good definition turns what could be a useful guide- 
post into a useless circular statement: models work 
in normal markets, and normal markets are defined 
as those within which models work. 

A better definition of a normal market is one in which 
the assumption mentioned above holds: institutions' 
actions do not themselves affect the market, and all 
market participants are purely victims of a set of ex- 
ogenous price processes. In this regime, it is reason- 
able to expect that the historical data is sufficient as a 
basis for forecasting, and that VaR models will pro- 
vide timely and accurate indicators of risk.* 

To illustrate what we would call a non-normal 
market, consider the dislocation in tranched credit 
derivatives in 2005. The first two tranches of the 
North American credit derivatives index (CDX ) typ- 
ically exhibit a very light relationship, with corre- 
lations in the 80-90% range. There is a funda- 
mental reason for this, as the two tranches repre- 
sent protection on the same underlying portfolio, al- 
beit against different levels of loss. In the first two 
weeks of May 2005. however, the tranches moved 


significantly against each other, without the price of 
the underlying index moving much, causing heavy 
losses for investors with (seemingly) hedged posi- 
tions across the two tranches. 

This was a case where no model based on historical 
data would have foreseen the losses to come. Though 
a savvy risk manager would have known that the re- 
lationship between the two tranches was not perfect, 
and that such a dislocation was possible in theory, no 
historical precedent existed for the magnitude of the 
losses. This is not. however, a case where we shnig 
our shoulders and mumble about fat tails. 

In the aftermath, it became clear that there had been 
a large buildup in positions that were short protection 
on the first tranche and long protection on the second. 
In such a position, a trader would reap a net quar- 
terly premium and be hedged against moves in the 
underlying portfolio, or so it seemed. But with many 
investors in the same trade, the inevitable happened. 
Sparked by an event external to the tranche market 
(likely the earlier downgrades of Ford and GM). an 
initial set of investors closed their positions, push- 
ing the prices of the two tranches apart and causing 
mark-to-market losses for those still holding the po- 
sition; this sparked more position closing, which led 
to greater losses, and so on. In the end. this was a 
classic example of a crowded trade. 2 3 

But with no historical precedent, was there anything 
risk managers could do? Possibly. Market mak- 
ers in these derivatives, themselves advocates for the 
"hedged" trade, could see the market flows leading 
to the crowded trade, prompting perhaps an invest i- 


2 To be clear, we use the word normal here in its generic sense, meaning typical or regular. Market returns could 
well be non-normally distributed; all we are claiming is that useful statistical inference from historical data is possible. 

3 See Finger (2005) for more detailed analysis. 
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gallon of (heir own positioning and susceptibility to 
a rush-to-the-exits scenario. Outside of flow desks, 
in 2005, then.* may have been nothing but market ru- 
mors to provide such insight. Today, however, the 
situation has improved. 

The Depository Trust Clearing Corporal ion (DTCC) 
began in the fall of 2008 to provide weekly snap- 
shots of the market exposures in credit default swaps, 
credit indices and tranches. For some of these prod- 
ucts, the reports are granular enough to note changes 
in exposure on individual contracts. For tranches, 
this is not yet the case, but it is not far fetched to be- 
lieve the members of the DTCC will agree to make 
this level of detail available in the future. Notably, 
this is true for an over-the-counter market which does 
not (yet) operate with a central counterparty. 

In short, the FSA is correct to criticize the current 
generation of VaR models for their inability to un- 
cover systemic risk or contagion effects. The data 
and tools to uncover such effects are closer than we 
might think, and should be the subject of significant 
attention in any regulatory review. 

A new backtest 

One year ago. we wrote in praise of the SSG for rec- 
ognizing the description of positions m a risk model 
as just as crucial as the volatility model or distri- 
butional assumption. Neither FSA document makes 
enough mention of this point. This is the second ca- 
sualty of the fat-tailed distraction. 


This can take the form of a missing risk factor 
assuming the basis between two similar instruments 
or the spread on a risky bond is constant — or a poor 
proxy choice — utilizing corporate bond yields to de- 
scribe the risk of a securitization with comparable 
rating. As the SSG pointed out in early 2008. and as 
conlmued to be the case, it was modeling the wrong 
(or no) risk factors that was the root of the worst un- 
derstatements of risk, not the choice of the wrong 
statistical distribution. The lack of coverage of credit 
risk in the trading book — what the IRC proposes to 
address — is a version of this same problem. 

There are statistical arguments to tell us how many 
exceptions we should see in theory, assuming VaR 
models work as advertised. And there is a history 
of disclosure on simple, standard backtesting mea- 
sures. We have a sense for how many exceptions the 
industry experiences, and know for instance that at 
99'? confidence, two VaR exceptions in a year is ex- 
pected. eight is fair but slightly concerning and fifty 
is outrageous. 

We propose that the description of instruments for 
risk purposes be backtested in their own right, inde- 
pendently of the statistical models used to forecast 
changes, and that the risk community establish a set 
of simple benchmarks for these tests. One candidate 
for such a benchmark is to compare over lime the 
actual market price changes on an instrument to the 
price changes that would appear in the risk model, 
with whatever assumptions (proxy factors, constant 
spreads, linear price relationships) that might be en- 
tailed. Simple correlations of these changes, aver- 
aged across asset classes, would provide a first indi- 
cator of the quality of instrument representation. 


The most overlooked source of bad VaR forecasts is 
the failure to adequately describe trading positions. 


5 


Of course, this sort of backtesting is only possible if 



104 


RbkMetncs Group 


Research Monthly 


we have actual market prices to compare. The ab- 
sence of such prices, and the inability to perform the 
proposed backtesting, could be used as criteria to ex- 
clude instruments from a model-based capital frame- 
work. or from regulatory trading book consideration. 

This is essentially the argument used (lack of prices, 
unclear pricing models and risk factors) for exclud- 
ing securitizations from trading book consideration, 
both in the FSA's proposed capital review and the 
Basel Committee’s IRC proposal. We are in agree- 
ment with this specific decision at the present time, 
but recommend that regulators and industry formal- 
ize what is expected for securitizations, or any instru- 
ment. to earn trading book treatment in the future. 

Moving to procylicality 

The last of the VaR deficiencies is that the procycli- 
cality of VaR contributed to excessive risk taking 
in the period prior to the crisis. A procyclical cap- 
ital framework is one that reinforces business cy- 
cles, requiring less capital when times appear good- 
encouraging greater risk taking and more capital 
as the economy contracts constraining banks' abil- 
ity to lend and working against economic growth. 
Alongside this point, there is a demonstration that 
VaR based on short histories will produce such pro- 
cyclical capital requirements. ‘ITie solution is to use 
longer historical periods, exactly as UBS did. Once 
again, the symptom is correct, but not the diagnosis. 


effort to embed into VaR the properties that are desir- 
able of regulatory capital. The call for longer obser- 
vation periods in order to eliminate procyclicality is 
a continuation of this mindset. So is the recent pro- 
posal of the Basel Committee to apply risk forecasts 
from a turbulent period to the positions of today, in 
order to calculate a so-called stressed VaR. 

The restrictions on VaR have not worked in the 
past — trading book capital under the current regime 
is still flawed — and further restrictions are unlikely 
to make it work in die future. At the same time, 
the UBS example demonstrates that VaR models that 
hew closely to regulatory desires perform poorly as 
risk forecasters. The effort to make a desirable cap- 
ital rule out of a good risk forecasting model has re- 
sulted in something that is neither. 

Looking to the horizon 

The bottom line is that short-horizon risk forecasts 
should be procyclical, and efforts to dampen this pro- 
duce worse forecasts. But two weeks is too short 
a horizon over which to set prudential capital. The 
horizons over which procyclicality matters are mea- 
sured in quarters or years, not days or weeks. For an 
institution or a system to build up risks over this type 
of horizon, it is not enough for a port folio at a spe- 
cific point in time to go bad (something VaR might 
wam against); rather, the systemic risks are a result 
of institutions’ reactions to an evolving market. 


The conflict between the needs of capital and the out- So the fundamental question for the fundamental re- 
put of VaR has existed since VaR first became part of view is how to detine minimum capital for trading 
the capital regime. 1'he regulators have responded books such that the regime is risk sensitive, is coun- 
lo this by placing restrictions on VaR models, in an tercyclical (or at least, not procyclical) and protects 
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nol jus I a fixed portfolio, but rather an institution as a 
going concern. While VaR on its own satisfies the 
first of these requirements, it is incompatible with 
the second, and by construction inappropriate for the 
third. There is a strong argument, then, for moving 
away from a capital requirement defined as a multi- 
ple of VaR, not because VaR fails to perform as ad- 
vertised. but because it succeeds. 

If we accept VaR for what it is, there may be a role 
for the measure in a new capital regime, even if nol 
the central role it plays today. And if not, the regu- 
lators should still emphasize that institutions should 
measure, manage and disclose the short-term risks 
in their trading portfolios. In either case, regula- 
tors should allow banks the freedom to build mod- 
els that best forecast short-term risk. In return, banks 
should nol only demonstrate adequate statistical per- 
formance of their risk forecasts, but also an accept- 
able link between any position's representation in the 
risk model and its actual market prices. 

Finally, some short-term events can in fact produce 
systemic risks: crowded trades, contagion, liquidity 
shocks, the transition to non-normal markets. Sig- 
nals for these should become part of the supervisory 


arsenal. But far from giving up and bucketing all 
of these as unknowable fat tails, we should be seek- 
ing indicators of when market imbalances have made 
these dynamics more likely. Regulators, banks and 
the rest of the financial community should share in 
this task. 

Further reading 

• C ampbell (2009). The Risk of Value-al-Risk. 
Risk. 22(4): 42-46. 

• Financial Services Authority (2009). A Regu- 
latory Response to the Global Banking Crisis. 
Discussion Paper 09/2. March. 

• Financial Services Authority (2009). The 
Turner Review: A Regulatory Response to the 
Global Banking Crisis. March. 

• Finger (2005). Eating Our Own Cooking, 
RiskMetrics Research Monthly, June. 

• Finger (2008). The SSG and My Two Brains. 
RiskMetrics Research Monthly, March. 


7 


Biography for Gregg E. Berman 

Gregg E. Berman, 43, is currently head of RiskMetrics Risk Business covering in- 
stitutional and wealth management offerings that serve Hedge Funds, Asset Man- 
agers, Prime Brokers, Banks, Financial Advisors, Insurance Companies, and 
Corporates. Mr. Berman joined RiskMetrics as a founding member during the time 
of its spin-off from J.P. Morgan in 1998 and has held a number of roles from re- 
search to head of product management, market risk, and of business management. 

Prior to joining RiskMetrics Group, Mr. Berman co-managed a number of multi- 
asset Hedge Funds within New York-based ED&F Man. His start in the Hedge 
Fund space began in 1993, researching and developing multi-asset trading strate- 



106 


gies as part of Mint Investment Management Corporation, a $lbn CTA based in 
New Jersey. 

Mr. Berman is a physicist by training and holds degrees from Princeton Univer- 
sity (Ph.D. 1994, M.S. 1989), and the Massachusetts Institute of Technology (B.S. 
1987). 

Chairman Miller. Thank you, Dr. Berman. 

Mr. Rickards for five minutes. 

STATEMENT OF MR. JAMES G. RICKARDS, SENIOR MANAGING 

DIRECTOR FOR MARKET INTELLIGENCE, OMNIS, INC., 

MCLEAN, VA 

Mr. Rickards. Mr. Chairman, my name is James Rickards and 
I appreciate the opportunity to speak to you on a subject of the ut- 
most importance to global capital markets. 

The world is two years into the worst financial crisis since the 
Great Depression. The list of culprits is long, including mortgage 
brokers, investment bankers and rating agencies. The story sadly 
is, by now, well known. What is less well known is that behind 
these actors were quantitative risk models which said that all was 
well even as the bus was driving over a cliff. 

Unfortunately, we have been here before. In 1998, capital mar- 
kets came to the brink of collapse due to the failure of a hedge 
fund, Long-Term Capital Management. The amounts involved seem 
small compared to today’s catastrophe. However, it did not seem 
that way at the time. I know, I was general counsel of LTCM. 
What is most striking to me now as I look back is how nothing has 
changed and how no lessons were learned. The lessons should have 
been obvious. LTCM used fatally flawed VaR models, too much le- 
verage, and the solutions were clear. Risk models needed to be 
changed or abandoned, leverage needed to be reduced, and regu- 
latory oversight needed to be increased. 

Amazingly, the United States Government did the opposite. They 
repealed Glass-Steagall in 1999 and allowed banks to act like 
hedge funds. The Commodity Futures Modernization Act of 2000 al- 
lowed more unregulated derivatives. SEC regulations in 2004 al- 
lowed increased leverage. It was as if the United States had looked 
at the catastrophe of LTCM and decided to double down. None of 
this would have happened without the assurance and comfort pro- 
vided to regulators and Wall Street by VaR models. But all models 
are based on assumptions. If the assumptions are flawed, no 
amount of mathematics will compensate. Therefore, the root of our 
inquiry into VaR should be an examination of the assumptions be- 
hind the models. 

The key assumptions are the following: one, the efficient market 
hypothesis, which assumes that investors behave rationally; two, 
the random walk, which assumes that no investor can beat the 
market consistently, because future prices are independent of the 
past; three, normally distributed risk. This says that since future 
price movements are random, the relationship of the frequency and 
the severity of the events will also be random, like a coin toss or 
roll of the dice. The random distribution is represented as a bell 
curve. Value-at-Risk would be a fine methodology but for the fact 
that all three of these assumptions are wrong. Markets are not effi- 
cient, future prices are not independent of the past, risk is not nor- 
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mally distributed. As the saying goes, “Besides that, Mrs. Lincoln, 
how was the play?” 

Behavioral economics has done a masterful job of showing that 
investors do not behave rationally and are guided by emotion. Simi- 
larly, prices do not move randomly but are dependent on past 
prices. In effect, news may be ignored for sustained periods of time 
until a kind of tipping point is achieved, at which point investors 
will react en masse. The normal distribution of risk has been 
known to be false since the early 1960s, when studies showed price 
distributions to be shaped in what is known as a power curve. A 
power curve has fewer low-impact events than the bell curve but 
has far more high-impact events. In short, a power curve cor- 
responds to market reality while a bell curve does not. 

Power curves have low predictability but can offer other valuable 
insights. One lesson is that as you increase the scale of the system, 
the size of the largest possible catastrophe grows exponentially. An 
example will illustrate the relationship between the scale of the 
system and the greatest catastrophe possible. Imagine a vessel 
with a large hold divided into three sections, separated by water- 
tight bulkheads. If a hole is punched in one section and that sec- 
tion fills with water, the vessel will still float. Now imagine the 
bulkheads are removed and the same hole is punched into the ves- 
sel. The entire hold will fill with water and the vessel will sink. In 
this example, the hold can be thought of as the system. The sinking 
of the vessel represents the catastrophic failure of the system. 
When the bulkheads are in place, we have three small systems. 
When the bulkheads are removed, we have one large system. By 
removing the bulkheads, we increase the scale of the system by a 
factor of three, but the likelihood of failure did not increase by a 
factor of three. It went from practically zero to practically 100 per- 
cent. The system size tripled, but the risk of sinking went up expo- 
nentially. 

If scale is the primary determinant of risk in complex systems, 
it follows that descaling is the most effective way to manage risk. 
This does not mean that the totality of the system needs to 
shrink — merely that it be divided into subcomponents with limited 
interaction. This has the same effect as installing the watertight 
bulkheads referred to above. In this manner, severe financial dis- 
tress in one sector does not result in contagion among all sectors. 

This descaling can be accomplished with three reforms: number 
one, the enactment of a modernized version of Glass-Steagall with 
a separation between bank deposit taking on the one hand, and 
market risk on the other; two, strict requirements for all derivative 
products to be traded on exchanges subject to margin position lim- 
its, price transparency and netting; three, higher regulatory capital 
requirements and reduced leverage for banks and brokers. Tradi- 
tional ratios of eight to one for banks and 15 to one for brokers 
seem adequate, provided off-balance sheet positions are included. 

Let us abandon VaR and the bell curve once and for all and ac- 
celerate empirical research into the actual metrics of event dis- 
tributions. Even if predictive value is low, there is value in know- 
ing the limits of our knowledge. Understanding the way risk me- 
tastasizes with scale might be lesson enough. It would offer a prop- 
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er dose of humility to those trying to supersize banks and regu- 
lators. 

Thank you for this opportunity to testify. 

[The prepared statement of Mr. Rickards follows:] 

Prepared Statement of James G. Rickards 

The Risks of Financial Modeling: 

VaR and the Economic Meltdown 


Introduction 

Mr. Chairman, Mr. Ranking Member and Members of this subcommittee, my 
name is James Rickards, and I want to extend my deep appreciation for the oppor- 
tunity and the high honor to speak to you today on a subject of the utmost impor- 
tance in the management of global capital markets and the global banking system. 
The Subcommittee on Investigations and Oversight has a long and distinguished 
history of examining technology and environmental matters which affect the health 
and well-being of Americans. Today our financial health is in jeopardy and I sin- 
cerely applaud your efforts to examine the flaws and misuse in financial modeling 
which have contributed to the impairment of the financial health of our citizens and 
the country as a whole. 

As a brief biographical note, I am an economist, lawyer and author and currently 
work at Omnis, Inc. in McLean, VA where I specialize in the field of threat finance 
and market intelligence. My colleagues and I provide expert analysis of global cap- 
ital markets to members of the national security community including military, in- 
telligence and diplomatic directorates. My writings and research have appeared in 
numerous journals and I am an Op-Ed contributor to the Washington Post and New 
York Times and a frequent commentator on CNBC, CNN, Fox and Bloomberg. I was 
formerly General Counsel of Long-Term Capital Management, the hedge fund at the 
center of the 1998 financial crisis, where I was principal negotiator of the Wall 
Street rescue plan sponsored by the Federal Reserve Bank of New York. 

Summary: The Problem with VaR 

The world is now two years into the worst financial crisis since the Great Depres- 
sion. The IMF has estimated that the total lost wealth in this crisis so far exceeds 
$60 Trillion dollars, more than the cost of all of the wars of the 20th century com- 
bined. The list of causes and culprits is long including mortgage brokers making 
loans borrowers could not afford, investment bankers selling securities while antici- 
pating their default, rating agencies granting triple-A ratings to bonds which soon 
suffered catastrophic losses, managers and traders focused on short-term profits and 
bonuses at the expense of their institutions, regulators acting complacently in the 
face of growing leverage and imprudence and consumers spending and borrowing at 
non-sustainable rates based on a housing bubble which was certain to burst at some 
point. This story, sadly, is by now well known. 

What is less well-known is that behind all of these phenomena were quantitative 
risk management models which told interested parties that all was well even as the 
bus was driving over a cliff. Mortgage brokers could not have made unscrupulous 
loans unless Wall Street was willing to buy them. Wall Street would not have 
bought the loans unless they could package them into securities which their risk 
models told them had a low risk of loss. Investors would not have bought the securi- 
ties unless they had triple-A ratings. The rating agencies would not have given 
those ratings unless their models told them the securities were almost certain to 
perform as expected. Transaction volumes would not have reached the levels they 
did without leverage in financial institutions. Regulators would not have approved 
that leverage unless they had confidence in the risk models being used by the regu- 
lated entities. In short, the entire financial edifice, from borrower to broker to bank- 
er to investor to rating agency to regulator, was supported by a belief in the power 
and accuracy of quantitative financial risk models. Therefore an investigation into 
the origins, accuracy and performance of those models is not ancillary to the finan- 
cial crisis; it is not a footnote; it is the heart of the matter. Nothing is more impor- 
tant to our understanding of this crisis and nothing is more important to the task 
of avoiding a recurrence of the crisis we are still living through. 

Unfortunately, we have been here before. In 1998, western capital markets came 
to the brink of collapse, owing to the failure of a hedge fund, Long-Term Capital 
Management, and a trillion dollar web of counter-party risk with all of the major 
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banks and brokers at that time. Then Fed Chairman Alan Greenspan and Treasury 
Secretary Robert Rubin called it the worst financial crisis in over 50 years. The 
amounts involved and the duration of the crisis both seem small compared to today’s 
catastrophe, however, it did not seem that way at the time. Capital markets really 
did teeter on the brink of collapse; I know, I was there. As General Counsel of Long- 
Term Capital Management, I negotiated the bail out which averted an even greater 
disaster at that time. What is most striking to me now as I look back is how nothing 
changed and how no lessons were applied. 

The lessons were obvious at the time. LTCM had used fatally flawed VaR risk 
models. LTCM had used too much leverage. LTCM had transacted in unregulated 
over-the-counter derivatives instead of exchange traded derivatives. The solutions 
were obvious. Risk models needed to be changed or abandoned. Leverage needed to 
be reduced. Derivatives needed to be moved to exchanges and clearinghouses. Regu- 
latory oversight needed to be increased. 

Amazingly the United States Government did the opposite. The repeal of Glass- 
Steagall in 1999 allowed banks to act like hedge funds. The Commodities Futures 
Modernization Act of 2000 allowed more unregulated derivatives. The Basle II ac- 
cords and SEC regulations in 2004 allowed increased leverage. It was as if the 
United States had looked at the near catastrophe of LTCM and decided to double- 
down. 

What reason can we offer to explain this all-in approach to financial risk? Cer- 
tainly the power of Wall Street lobbyists and special interests cannot be discounted. 
Alan Greenspan played a large role through his belief that markets could self-regu- 
late through the intermediation of bank credit. In fairness, he was not alone in this 
belief. But none of this could have prevailed in the aftermath of the 1998 collapse 
without the assurance and comfort provided by quantitative risk models. These 
models, especially Value-at-Risk, cast a hypnotic spell, as science often does, and as- 
sured bankers, investors and regulators that all was well even as the ashes of 
LTCM were still burning. 

What are these models? What is the attraction that allows so much faith to be 
placed in them? And what are the flaws which lead to financial collapse time and 
time again? 

The term “Value-at-Risk” or VaR is used in two senses. One meaning refers to 
the assumptions, models and equations which constitute the risk management sys- 
tems most widely used in large financial institutions today. The other meaning re- 
fers to the output of those systems, as in, “our VaR today is $200 million” which 
refers to the maximum amount the institution is expected to lose in a single day 
within some range of probability or certainty usually expressed at the 99 percent 
level. For purposes of this testimony, we will focus on VaR in the first sense. If the 
models are well founded then the output should be of some value. If not, then the 
output will be unreliable. Therefore the proper focus of our inquiry should be on the 
soundness of the models themselves. 

Furthermore, any risk management system is only as good as the assumptions be- 
hind it. It seems fair to conclude that based on a certain set of assumptions, the 
quantitative analysts and computer developers are able within reason to express 
those assumptions in equations and to program the equations as computer code. In 
other words, if the assumptions are correct then it follows that the model develop- 
ment and the output should be reasonably correct and useful as well. Conversely, 
if the assumptions are flawed then no amount of mathematical equation writing and 
computer development will compensate for this deficiency and the output will al- 
ways be misleading or worse. Therefore, the root of our inquiry into models should 
be an examination of the assumptions behind the models. 

In broad terms, the key assumptions are the following: 

The Efficient Market Hypothesis (EMH): This assumes that investors and market 
participants behave rationally from the perspective of wealth maximization and will 
respond in a rational manner to a variety of inputs including price signals and 
news. It also assumes that markets efficiently price in all inputs in real time and 
that prices move continuously and smoothly from one level to another based on 
these new inputs. 

The Random Walk: This is a corollary to EMH and assumes that since markets effi- 
ciently price in all information, no investor can beat the market consistently because 
any information which an investor might rely on to make an investment decision 
is already reflected in the current market price. This means than future market 
prices are independent of past market prices and will be based solely on future 
events that are essentially unknowable and therefore random. 
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Normally Distributed Risk: This is also a corollary to EMH and says that since fu- 
ture price movements are random, their degree distribution (i.e., relationship of fre- 
quency to severity of events) will also be random like a coin toss or roll of the dice. 
This random or normal degree distribution is also referred to as Gaussian and is 
most frequently represented as a bell curve in which the large majority of outcomes 
are bunched in a region of low severity with progressively fewer outcomes shown 
in the high severity region. Because the curve tails off steeply, highly extreme 
events are so rare as to be almost impossible. 

Value-at-Risk would be a fine methodology but for the fact that all three of these 
assumptions are wrong. Markets are not efficient. Future prices are not independent 
of the past. Risk is not normally distributed. As the saying goes, “Besides that, Mrs. 
Lincoln, how was the play?” Let’s take these points separately. 

Behavioral economics has done a masterful job of showing experimentally and em- 
pirically that investors do not behave rationally and that markets are not rational 
but are prone to severe shocks or mood swings. Examples are numerous but some 
of the best known are risk aversion (i.e., investors put more weight on avoiding risk 
than seeking gains), herd mentality (i.e., investors buy stocks when others are buy- 
ing and sell when others are selling leading to persistent losses) and various sea- 
sonal effects. Prices do not smoothly and continuously move from one price level to 
the next but have a tendency to gap up or down in violent thrusts depriving inves- 
tors of the chance to get out before large losses are incurred. 

Similarly, prices to not move randomly but are highly dependent on past price 
movements. In effect, relevant news will be discounted or ignored for sustained peri- 
ods of time until a kind of tipping point is achieved at which point investors will 
react en masse to what is mostly old news mainly because other investors are doing 
likewise. This is why markets exhibit periods of low and high volatility in succes- 
sion, why markets tend to overshoot in response to fundamental news and why in- 
vestors can profit consistently by momentum trading which exploits an under- 
standing of these dynamics. 

Finally, the normal distribution of risk has been known to be false at least since 
the early 1960’s when published studies of time series of prices showed price dis- 
tributions to be shaped in what is known as a power curve rather than a bell curve. 
This has been borne out by many studies since. A power curve has fewer low impact 
events than the bell curve but has far more high impact events. This corresponds 
exactly to the actual market behavior we have seen including frequent extreme 
events such as the stock market crash of 1987, the Russian-LTCM collapse of 1998, 
the dot corn bubble collapse of 2000 and the housing collapse of 2007. Statistically 
these events should happen once every 1,000 years or so in a bell curve distribution 
but are expected with much greater frequency in a power curve distribution. In 
short, a power curve corresponds to market reality while a bell curve does not. 

How is it possible that our entire financial system has come to the point that it 
is risk managed by a completely incorrect system? 

The Nobelist, Daniel Kahneman, tells the story of a Swiss Army patrol lost in the 
Alps in a blizzard for days. Finally the patrol stumbles into camp, frostbitten but 
still alive. The Commander asks how they survived and the patrol leader replies, 
“We had a map.” The Commander looks at the map and says, “This is a map of 
the Pyrenees; you were in the Alps.” “Yes,” comes the reply; “but we had a map.” 
The point is that sometimes bad guidance is better than no guidance; it gives you 
confidence and an ability to function even though your system is flawed. 

So it is with risk management on Wall Street. The current system, based on the 
idea that risk is distributed in the shape of a bell curve, is flawed and practitioners 
know it. Practitioners treat extreme events as outliers and develop mathematical 
fixes. They call extreme events fat tails and model them separately from the rest 
of the bell curve. They use stress tests to gauge the impact of extreme events. The 
problem is they never abandon the bell curve. They are like medieval astronomers 
who believe the sun revolves around the earth and are furiously tweaking their geo- 
centric math in the face of contrary evidence. They will never get this right; they 
need their Copernicus. 

But the right map exists. It’s called a power curve. It says that events of any size 
can happen and extreme events happen more frequently than the bell curve pre- 
dicts. There is no need to treat fat tails as a special case; they occur naturally on 
power curves. And power curves are well understood by scientists because they 
apply to extreme events in many natural and man-made systems from power out- 
ages to earthquakes. 

Power curve analysis is not new. The economist, Vilfredo Pareto, observed in 1906 
that wealth distributions in every society conform to a power curve; in effect, there 
is one Bill Gates for every 100 million average Americans. Benoit Mandelbrot pio- 
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neered empirical analysis in the 1960’s that showed market prices move in power 
curve patterns. 

So why have we gone down the wrong path of random walks and normal distribu- 
tions for the past 50 years? The history of science is filled with false paradigms that 
gained followers to the detriment of better science. People really did believe the sun 
revolved around the earth for 2,000 years and mathematicians had the equations 
to prove it. The sociologist, Robert K. Merton, called this the Matthew Effect from 
a New Testament verse that says, “For to those who have, more will be given . . .” 
The idea is that once an intellectual concept attracts a critical mass of supporters 
it becomes entrenched while other concepts are crowded out of the marketplace of 
ideas. 

Another reason is that practitioners of bell curve science became infatuated with 
the elegance of their mathematical solutions. The Black-Scholes options formula is 
based on bell curve type price movements. The derivatives market is based on vari- 
ations of Black-Scholes. Wall Street has decided that the wrong map is better than 
no map at all — as long as the math is neat. 

Why haven’t scientists done more work in applying power curves to capital mar- 
kets? Some excellent research has been done. But one answer is that power curves 
have low predictive value. Researchers approach this field to gain an edge in trading 
and once the edge fails to materialize they move on. But the Richter Scale, a classic 
power curve, also has low predictive value. That does not make earthquake science 
worthless. We know that 8.0 earthquakes are possible and we build cities accord- 
ingly even if we cannot know when the big one will strike. 

We can use power curve analysis to make our financial system more robust even 
if we cannot predict financial earthquakes. One lesson of power curves is that as 
you increase the scale of the system, the risk of a mega-earthquake goes up expo- 
nentially. If you increase the value of derivatives by a factor of 10, you may be in- 
creasing risk by a factor of 10,000 without even knowing it. This is not something 
that Wall Street or Washington currently comprehend. 

Let’s abandon the bell curve once and for all and accelerate empirical research 
into the proper risk metrics of event distributions. Even if predictive value is low, 
there is value in knowing the limits of our knowledge. Understanding the way risk 
metastasizes with scale might be lesson enough. It would offer a proper dose of hu- 
mility to those trying to supersize banks and regulators. 

Detailed Analysis — History of VaR Failures 

The empirical failures of the Efficient Market Hypothesis and VaR are well 
known. Consider the October 19, 1987 stock market crash in which the market fell 
22.6 percent in one day; the December 1994 Tequila Crisis in which the Mexican 
Peso fell 85 percent in one week; the September 1998 Russian-LTCM crisis in which 
capital markets almost ceased to function; the March 2000 dot corn collapse during 
which the NASDAQ fell 80 percent over 30 months, and the 9-11 attacks in which 
the NYSE first closed and then fell 14.3 percent in the week following its reopening. 
Of course, to this list of extreme events must now be added the financial crisis that 
began in July 2007. Events of this extreme magnitude should, according to VaR, ei- 
ther not happen at all because diversification will cause certain risks to cancel out 
and because rational buyers will seek bargains once valuations deviate beyond a 
certain magnitude, or happen perhaps once every 1,000 years (because standard de- 
viations of this degree lie extremely close to the x-axis on the bell curve which cor- 
responds to a value close to zero on the y-axis, i.e., an extremely low frequency 
event). The fact that all of these extreme events took place in just over 20 years 
is completely at odds with the predictions of VaR in a normally distributed para- 
digm. 

Practitioners treated these observations not as fatal flaws in VaR but rather as 
anomalies to be explained away within the framework of the paradigm. Thus was 
born the “fat tail” which is applied as an embellishment on the bell curve such that 
after approaching the x-axis (i.e., the extreme low frequency region), the curve flat- 
tens to intersect data points representing a cluster of highly extreme but not so 
highly rare events. No explanation is given for what causes such events; it is simply 
a matter of fitting the curve to the data (or ignoring the data) and moving on with- 
out disturbing the paradigm. This process of pinning a fat tail on the bell curve 
reached its apotheosis in the invention of generalized auto-regressive conditional 
heteroskedasicity or GARCH and its ilk, which are analytical techniques for mod- 
eling the section of the degree distribution curve containing the extreme events as 
a separate case and feeding the results of this modeling into a modified version of 
the curve. A better approach would have been to ask the question: if a normal dis- 
tribution has a fat tail, is it really a normal distribution? 
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A Gaussian distribution is not the only possible degree distribution. One of the 
most common distributions in nature, which accurately describes many phenomena, 
is the power curve which shows that the severity of an event is inversely propor- 
tional to its frequency with the proportionality expressed as an exponent. When 
graphed on a double logarithmic scale, the power law describing financial markets 
risk is a straight line sloping downward from left to right; the negative exponent 
is the slope of the line. 

This difference is not merely academic. Gaussian and power curve distributions 
describe two entirely different phenomena. Power curves accurately describe a class 
of phenomena known as nonlinear dynamical systems which exhibit scale 
invariance, i.e., patterns are repeated at all scales. 

The field of nonlinear dynamical systems was enriched in the 1990s by the con- 
cept of self-organized criticality. The idea is that actions propagate throughout sys- 
tems in a critical chain reaction. In the critical state, the probability that an action 
will propagate is roughly balanced by the probability that the original action will 
dissipate. In the subcritical state, the probability of extensive effects from the initial 
action is low. In the super-critical state, a single minor action can lead to a cata- 
strophic collapse. Such states have long been observed in physical systems, e.g., nu- 
clear chain reactions in uranium piles, where a small amount of uranium is rel- 
atively harmless ( subcritical ) and larger amounts can either be carefully controlled 
to produce desired energy (critical), or can be shaped to produce atomic explosions 
(supercritical). 

The theory of financial markets existing in a critical state cannot be tested in a 
laboratory or particle accelerator in the same fashion as theories of atomic physics. 
Instead, the conclusion that financial markets are a nonlinear critical state system 
rests on two non-experimental bases; one deductive, one inductive. The deductive 
basis is the ubiquity of power curves as a description of the behavior of a wide vari- 
ety of complex systems in natural and social sciences, e.g., earthquakes, forest fires, 
sunspots, polarity, drought, epidemiology, population dynamics, size of cities, wealth 
distribution, etc. This is all part of a more general movement in many natural and 
social sciences from 19th and early 20th century equilibrium models to non-equi- 
librium models; this trend has now caught up with financial economics. 

The inductive basis is the large variety of capital markets behavior which has 
been empirically observed to fit well with the nonlinear paradigm. It is certainly 
more robust than VaR when it comes to explaining the extreme market movements 
described above. It is consistent with the fact that extreme events are not nec- 
essarily attributable to extreme causes but may arise spontaneously in the same ini- 
tial conditions from routine causes. 

While extreme events occur with much greater than normal frequency in non- 
linear critical state systems, these events are nevertheless limited by the scale of 
the system itself. If the financial system is a self-organized critical system, as both 
empirical evidence and deductive logic strongly suggest, the single most important 
question from a risk management perspective is: what is the scale of the system ? 
Simply put, the larger the scale of the system, the greater the potential collapse 
with correlative macroeconomic and other real world effects. 

The news on this front is daunting. There is no normalized scale similar to the 
Richter Scale for measuring the size of markets or the size of disruptive events that 
occur within them, however, a few examples will make the point. According to re- 
cent estimates prepared by the McKinsey Global Institute, the ratio of world finan- 
cial assets to world GDP grew from 100 percent in 1980 to 200 percent in 1993 to 
316 percent in 2005. Over the same period, the absolute level of global financial as- 
sets increased from $12 trillion to $140 trillion. The drivers of this exponential in- 
crease in scale are globalization, derivative products, and leverage. 

Globalization in this context is the integration of capital markets across national 
boundaries. Until recently there were specific laws and practices that had the effect 
of fragmenting capital markets into local or national venues with little interaction. 
Factors included withholding taxes, capital controls, protectionism, non-convertible 
currencies, licensing, regulatory and other restrictions that tilted the playing field 
in favor of local champions and elites. All of these impediments have been removed 
over the past 20 years to the point that the largest stock exchanges in the United 
States and Europe (NYSE and Euronext) now operate as a single entity. 

Derivative products have exhibited even faster growth than the growth in under- 
lying financial assets. This stems from improved technology in the structuring, pric- 
ing, and trading of such instruments and the fact that the size of the derivatives 
market is not limited by the physical supply of any stock or commodity but may 
theoretically achieve any size since the underlying instrument is notional rather 
than actual. The total notional value of all swaps increased from $106 trillion to 
$531 trillion between 2002 and 2006. The notional value of equity derivatives in- 
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creased from $2.5 trillion to $11.9 trillion over the same period while the notional 
value of credit default swaps increased from $2.2 trillion to $54.6 trillion. 

Leverage is the third element supporting the massive scaling of financial markets; 
margin debt of U.S. brokerage firms more than doubled from $134.58 billion to 
$293.2 billion from 2002 to 2007 while the amount of total assets per dollar of eq- 
uity at major U.S. brokerage firms increased from approximately $20 to $26 in the 
same period. In addition, leveraged investors invest in other entities which use le- 
verage to make still further investments. This type of layered leverage is impossible 
to unwind in a panic. 

There can be no doubt that capital markets are larger and more complex than 
ever before. In a dynamically complex critical system, this means that the size of 
the maximum possible catastrophe is exponentially greater than ever. Recalling that 
systems described by a power curve allow events of all sizes and that such events 
can occur at any time, particularly when the system is super-critical, the conclusion 
is inescapable that progressively greater financial catastrophes of the type we are 
experiencing today should be expected frequently. 

The more advanced risk practitioners have long recognized the shortcomings of 
using VaR in a normally distributed paradigm to compute risk measured in stand- 
ard deviations from the norm. This is why they have added stress testing as an al- 
ternative or blended factor in their models. Such stress testing rests on historically 
extreme events such as the market reaction to 9-11 or the stock market crash of 
1987. However, this methodology has its own flaws since the worst outcomes in a 
dynamically complex critical State system are not bounded by history but are only 
bounded by the scale of the system itself. Since the system is larger than ever, there 
is nothing in historical experience that provides a guide to the size of the largest 
catastrophe that can arise today. The fact that the financial crisis which began in 
July 2007 has lasted longer, caused greater losses and been more widespread both 
geographically and sectorally than most analysts predicted or can explain is because 
of the vastly greater scale of the financial system which produces an exponentially 
greater catastrophe than has ever occurred before. This is why the past is not a 
guide and why the current crisis may be expected to produce results as severe as 
the Great Depression of 1929-1941. 

Policy Approaches and Recommendations 

A clear understanding of the structures and vulnerabilities of the financial mar- 
kets points the way to solutions and policy recommendations. These recommenda- 
tions fall into the categories of limiting scale, controlling cascades, and securing in- 
formational advantage. 

To explain the concept of limiting scale, a simple example will suffice. If the U.S. 
power grid east of the Mississippi River were at no point connected to the power 
grid west of the Mississippi River, a nationwide power failure would be an ex- 
tremely low probability event. Either the “east system” or the “west system” could 
fail catastrophically in a cascading manner but both systems could not fail simulta- 
neously except for entirely independent reasons because there are no nodes in com- 
mon to facilitate propagation across systems. In a financial context, governments 
should give consideration to preventing mergers that lead to globalized stock and 
bond exchanges and universal banks. The first order efficiencies of such mergers are 
outweighed by the risks of large-scale failure especially if those risks are not prop- 
erly understood and taken into account. 

Another example will help to illustrate the relationship between the scale of a sys- 
tem and extent of the greatest catastrophe possible in that system. Imagine a vessel 
with a large hold. The hold is divided into three equal sections separated by water- 
tight bulkheads. If a hole is punched in one section and that section is completely 
filled with water, the vessel will still float. Now imagine the watertight bulkheads 
are removed and the same hole is punched into the vessel. In this case, the entire 
hold will fill with water and the vessel will sink. In this example, the area of the 
hold can be thought of as the relevant dynamic system. The sinking of the vessel 
represents the catastrophic failure of the system. When the bulkheads are in place 
we have three small systems. When the bulkheads are removed we have one large 
system. By removing the bulkheads we increased the scale of the system by a factor 
of three. But the likelihood of failure did not increase by a factor of three; it went 
from practically zero to practically 100 percent. The system size tripled but the risk 
of sinking went up exponentially. By removing the bulkheads we created what engi- 
neers call a “single point of failure,” i.e., one hole is now enough to sink the entire 
vessel. 

Something similar happened to our financial system between 1999 and 2004. This 
began with the repeal of Glass-Steagall in 1999 which can be thought of as remov- 
ing the watertight bulkheads separating commercial banks and investment banks. 
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This was exacerbated by the Commodities Futures Modernization Act of 2000 which 
removed the prohibition on many kinds of derivatives. This allowed banks to in- 
crease the scale of the system through off-balance sheet transactions. Finally, in 
2004, the SEC amended the broker-dealer net capital rule in such a way that al- 
lowed brokers to go well-beyond the traditional 15:1 leverage ratio and to use lever- 
age of 30:1 or more. All three of these events increased the scale of the system by 
allowing regulated financial institutions to enter new markets, trade new products 
and use increased leverage. Using a power curve analysis, we see that while the 
scale of the system was increased in a linear way (by a factor of three, five, ten or 
fifty depending on the product) the risk was increasing in a nonlinear way (by a 
factor of 100, 1000, or 10,000 depending on the slope of the power curve). VaR mod- 
els based on normal distributions were reporting that risk was under control and 
sounding the all clear signal because so much of the risk was offsetting or seen to 
cancel out in the models. However, a power curve model would have been flashing 
a red alert sign because it does not depend on correlations, instead it sees risk as 
an emergent property and an exponential function of scale. 

The fact that government opened the door to instability does not necessarily mean 
that the private sector had to rush through the door to embrace the brave new 
world of leveraged risk. For that we needed VaR. Without VaR models to tell bank- 
ers that risk was under control, managers would not have taken so much risk even 
if government rules allowed them to do so. Self-interest would have constrained 
them somewhat as Greenspan expected. But with VaR models telling senior man- 
agement that risk was contained the new government rules became an open invita- 
tion to pile on massive amounts of risk which bankers promptly did. 

Our financial system was relatively stable from 1934-1999 despite occasional fail- 
ures of institutions (such as Continental Illinois Bank) and entire sectors (such as 
the S&L industry). This 65-year period can be viewed as the golden age of compart- 
mented banking and moderate leverage under Glass-Steagall and the SEC’s original 
net capital rule. Derivatives themselves were highly constrained by the Commodity 
Exchange Act. In 1999, 2000 and 2004 respectively, all three of these watertight 
bulkheads were removed. By 2006 the system was poised for the most catastrophic 
financial collapse in history. While subprime mortgage failures provided the cata- 
lyst, it was the scale of the system itself which caused the damage. The catalyst 
could just as well have come from emerging markets, commercial real estate or cred- 
it default swaps. In a dynamically critical system, the catalyst is always less impor- 
tant than the chain reaction and the reaction in this case was a massive collapse. 

The idea of controlling cascades of failure is, in part, a matter of circuit breakers 
and pre-rehearsed crisis management so that nascent collapses do not spin into full 
systemic catastrophes before regulators have the opportunity to prevent the spread. 
The combination of diffuse credit and layered leverage makes it infeasible to assem- 
ble all of the affected parties in a single room to discuss solutions. There simply is 
not enough time or condensed information to respond in real time as a crisis 
unfolds. 

One significant circuit breaker which has been discussed for over a decade but 
which has still not been fully implemented is a clearinghouse for all over-the- 
counter derivatives. Experience with clearinghouses and netting systems such as the 
Government Securities Clearing Corporation shows that gross risk can be reduced 
90 percent or more when converted to net risk through the intermediation of a 
clearinghouse. Bearing in mind that a parametric decrease in scale produces an ex- 
ponential decrease in risk in a nonlinear system, the kind of risk reduction that 
arises in a clearinghouse can be the single most important step in the direction of 
stabilizing the financial system today; much more powerful than bail outs which do 
not reduce risk but merely bury it temporarily. 

A clearinghouse will also provide informational transparency that will allow regu- 
lators to facilitate the failure of financial institutions without producing contagion 
and systemic risk. Such failure (what Joseph Schumpeter called “creative destruc- 
tion”) is another necessary step on the road to financial recovery. Technical objec- 
tions to clearinghouse implementation based on the non-uniformity of contracts can 
be overcome easily through consensual contractual modification with price adjust- 
ments upon joining the clearinghouse enforced by the understanding that those who 
refuse to join will be outside the safety net. Only by eliminating zombie institutions 
and creating breathing room for healthy institutions with sound balance sheets can 
the financial sector hope to attract sufficient private capital to replace government 
capital and thus re-start the credit creation process needed to produce sound eco- 
nomic growth. 

Recently a number of alternative paradigms have appeared which not only do not 
rely on VaR but rather assume its opposite and build models that are more robust 
to empirical evidence and market price patterns. Several of these approaches are: 
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Behavioral Economics — This field relies on insights into human behavior derived 
from social science and psychology, in particular, the “irrational” nature of human 
decision-making when faced with economic choices. Insights include risk aversion, 
herding, the presence or absence of cognitive diversity and network effects among 
others. While not summarized in a general theory and while not always amendable 
to quantitative modeling, the insights of behavioral economics are powerful and 
should be considered in weighing reliance on VaR-style models which do not make 
allowance for subjective influences captured in this approach. 

Imperfect Knowledge Economics — This discipline (under the abbreviation IKE) 
attempts to deal with uncertainty inherent in capital markets by using a combina- 
tion of Bayesian networks, link analysis, causal inference and probabilistic 
hypotheses to fill in unknowns using the known. This method is heavily dependent 
on the proper construction of paths and the proper weighing of probabilities in each 
hypothesis cell or evidence cell, however, used properly it can guide decision-making 
without applying the straitjacket of VaR. 

Econoahysics — This is a branch of financial economics which uses insights gained 
from physics to model capital markets behavior. These insights include nonlinearity 
in dynamic critical state systems the concept of phase transitions. Such systems ex- 
hibit an unpredictably deterministic nonlinear relationship between inputs and out- 
puts (the so-called “Butterfly Effect”) and scale invariance which accords well with 
actual time series of capital markets prices. Importantly, this field leads to a degree 
distribution characterized by the power curve rather than the bell curve with impli- 
cations for scaling metrics in the management of systemic risk. 

It may be the case that these risk management tools work best at distinct scales. 
For example, behavioral economics seems to work well at the level of individual de- 
cision-making but has less to offer at the level of the system as a whole where com- 
plex feedback loops cloud its efficacy. IKE may work best at the level of a single 
institution where the hypothesis and evidence cells can be reasonably well defined 
and populated. Econophysics may work best at the systemic level because it goes 
the furthest in its ability to model highly complex dynamics. This division of labor 
suggests that rather than replacing VaR with a one-size-fits-all approach, it may be 
best to adopt a nested hierarchy of risk management approaches resembling the fol- 
lowing: 


Econophysics 

Normalized metrics for understanding systemic risk 


Imperfect Knowledge Economics 

Processes to aid risk and resource allocation at the enterprise level 


Behavioral Economics 

Experimentally derived tools to aid individual decision making 


While all of these approaches and others not mentioned here require more re- 
search to normalize metrics and build general theories, they are efficacious and ro- 
bust alternatives to EMH and VaR and their development and use can serve a sta- 
bilizing function since they have a strong empirical basis unlike EMH and VaR. 

In summary, Wall Street’s reigning risk management paradigm consisting of VaR 
using a normally distributed model combined with GARCH techniques applied to 
the non-normal region and stress testing to account for outliers is a manifest failure. 
It should be replaced at the systemic level with the empirically robust model based 
on nonlinear complexity and critical state dynamics as described by the power 
curve. This method also points the way to certain solutions, most importantly the 
creation of an over-the-counter derivatives clearinghouse which will de-scale the sys- 
tem and lead to an exponential decrease in actual risk. Such a clearinghouse can 
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also be used to improve transparency and manage failure in ways that can leave 
the system far healthier while avoiding systemic collapse. 

Importantly, if scale is the primary determinant of risk, as appears to be the case 
in complex systems such as the financial markets, then it follows that de-scaling the 
system is the simplest and most effective way to manage risk. This does not mean 
that the totality of the system needs to shrink, merely that it be divided into sub- 
components with limited interaction. This has the same effect as installing the wa- 
tertight bulkheads referred to in the example above. In this manner, severe finan- 
cial distress in one sector does not automatically result in contagion among all sec- 
tors. 

This effective de-scaling can be accomplished with three reforms: 

1. The enactment of a modernized version of Glass-Steagall with a strict separation 
between commercial banking and deposit taking on the one hand and principal 
risk taking in capital markets on the other. 

2. Strict requirements for all derivative products to be traded on exchanges subject 
to credit tests for firm memberships, initial margin, variation margin, position 
limits, price transparency and netting. 

3. Higher regulatory capital requirements and reduced leverage for banks and 
broker-dealers. Traditional ratios of 8:1 for banks and 15:1 for brokers seem ade- 
quate provided off-balance sheet positions (other than exchange traded contracts 
for which adequate margin is posted) be included for this purpose. 

These rules can be implemented directly and do not depend on the output of ar- 
cane and dangerous models such as VaR. Instead, they derive from another proven 
model, the power curve, which teaches that risk is an exponential function of scale. 
By de-scaling, we radically reduce risk and restore stability to individual institu- 
tions and to the system as a whole. 
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Chairman Miller. Thank you, Mr. Rickards. I did practice re- 
peatedly saying “Taleb.” I should have practiced “Rickards” as well. 

Mr. Whalen. 

STATEMENT OF MR. CHRISTOPHER WHALEN, MANAGING 
DIRECTOR, INSTITUTIONAL RISK ANALYTICS 

Mr. Whalen. Thank you, Mr. Chairman. I am going to just sum- 
marize a couple points further to my written testimony. You will 
notice in my comments I focused on the distinction between subjec- 
tivity and objectivity, and I think this committee is probably better 
placed to understand those distinctions than most of the other pan- 
els in the Congress. 

You know, we have seen over the last 100 years in this country 
a shift in our financial world from focusing on current performance 
of companies and financial institutions to focusing on predicting 
the future. This is very well illustrated in the Graham and Dodd 
volume, Securities Analysis, in chapter 38 where they talk about 
new era investing, and I urge you to reread that if you have never 
done so before. 

The bottom line to me as someone who has worked in the indus- 
try as a supervisor and a trader and investment banker, is that 
when you use assumptions and models, you have already stepped 
off the deep edge, you know, the deep end of the pool, and there’s 
no water in the pool. You essentially are in the world of specula- 
tion, and you have left the world of investing. Why do I say this? 
Well, if we use the same rules that govern the assumptions that 
go into most VaR models to design airplanes and buildings and 
dams, all of these physical structures would fail, because they vio- 
late the basic rules of scientific method that the Members of this 
committee know very, very well. I would submit to you that if we 
are going to allow our financial system to design products that are 
based on assumptions rather than hard data, than we are in big 
trouble. My firm has over the last seven years shunned the quan- 
titative world. Our entire methodology is focused on benchmarking 
the current performance of banks, and taking observations about 
that current performance that may suggest what they are going to 
do in the future. But we don’t guess, we don’t speculate. We have 
almost 20,000 retail customers who use the bank monitor to track 
the safety and soundness of their institution. It is an entirely me- 
chanical survey process. We stress-test every bank in the United 
States the same way, whether it is J.P. Morgan or Cullen/Frost 
Bank in Texas. We ask the same question, how did you do this 
quarter, and we compare it to 1995, which was a nice, boring year. 

The second point I would like to make is that I think a big part 
of the problem is that we allowed the economist profession to es- 
cape from the world of social sciences, and enter into an unholy 
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union with commission-driven dealers in the securities market. 
Your colleague, Mr. Broun, said earlier that economists can’t make 
up their mind. Well, yes, they can. When they are working in the 
securities business they have no trouble making up their mind. 
They offer opinions and hypotheses and ‘what if or ‘I want’ in re- 
gards to the creation of a security. This is a big problem. I wouldn’t 
let most economists park my car, and the problem is not that they 
are not smart people, not that they are not interesting people, but 
they live in the world of supposition rather than the world of fact, 
and again, their methodologies are not governed by the iron rules 
that you find in physics or chemistry or any of the other physical 
sciences where you have to live by those rules. You can’t come up 
with some neat concept and say to your colleagues, hey, look at me, 
or hey, look at this new CDO I designed, and then go out and sell 
that security to the public. 

I think it all comes down at the end of the day to what kind of 
economy do we want. There is an old-fashioned American concept 
called ‘fair dealing 5 that I spent a lot of time talking in my testi- 
mony to the Senate Banking Committee earlier this year, and it 
comes from the Greeks, the concept of proportional requital. One 
person gives value, the other person receives value. The problem 
with products like credit default swaps, is that they are entirely 
speculative. There is no visible underlying market for single-name 
credit default swaps really. The corporate bonds that are sup- 
posedly the derivative or the basis for the derivative are fairly liq- 
uid and not a very good source of pricing information, so we use 
models and we then sell these securities to anyone and everyone. 
I would submit that that is unfair, and it goes against the basic 
grain of American society that we are a fair and transparent na- 
tion. So bottom line to me is, if you want to fix the problem, I think 
we have got to reimpose not higher capital requirements on banks 
that are out of control, and which take risks that no one can really 
quantify. I think what we have to do is reimpose restrictions on 
their risk taking and get them to the point where an eight percent 
capital assets ratio makes sense again, because it clearly doesn’t 
now. Does anybody really think we can get the private sector to 
double the capital of J.P. Morgan when their equity returns are 
going to be falling for the next couple of years? The only entity that 
would fund that opportunity would be a government, so what we 
are really saying is that these are GSEs. I think we have got to 
come back almost to the Glass-Steagall-era draconian division be- 
tween the utility function of a bank and the transactional function 
of hedge funds, broker dealers, whatever, and that latter group can 
do whatever they want. 

So let me stop there, and I look forward to your questions. 

[The prepared statement of Mr. Whalen follows:] 

Prepared Statement of Christopher Whalen 

Chairman Miller, Congressman Broun, Members of the Committee, my name is 
Christopher Whalen and I live in the State of New York. I work in the financial 
community as an analyst and a principal of a firm that rates the performance of 
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commercial banks . 1 Thank you for inviting my comments today on this important 
subject. 

The Committee has asked witnesses to comment on the topic of “The Risks of Fi- 
nancial Modeling: VaR and the Economic Meltdown.” The comments below reflect 
my own views, as well as comments from my colleague and business partner Dennis 
Santiago, and others in the financial and risk management community. 

By way of background, our firm provides ratings for assessing the financial condi- 
tion of U.S. banks and commercial companies. We build the analytical tools that we 
use to support these rating activities and produce reports for thousands of consumer 
and professional users. 

We use mathematical tools such as models to explore the current financial behav- 
ior of a given subject. In the course of our work, we use these tools to make esti- 
mates, for example, as to the maximum probable loss in a bank’s loan portfolio 
through an economic cycle or the required Economic Capital for a financial institu- 
tion. Models help us understand and illustrate how the financial condition of a bank 
or other obliger have changed and possibly will change in the future. 

But in all that we at Institutional Risk Analytics do in the world of ratings and 
financial analysis, we do our best to separate objective measures based upon empir- 
ical observations, and subjective analyses that employ speculative assumptions and 
directives which are often inserted into the very ground rules for the analysis proc- 
ess itself. The difference between subjectivity and objectivity in finance has signifi- 
cant implications for national policy when it comes to financial markets and institu- 
tions. 

I strongly suggest to the Committee that they bear the distinction between objec- 
tive and subjective measures in mind when discussing the use of models in finance. 
Obtaining a better understanding of the role of inserting subjectivity into models 
is critical for distinguishing between useful deployments of modeling to manage risk 
and situations where models are the primary failure pathway towards creating sys- 
temic risk and thus affect economic stability and public policy. 

Used as both a noun and a verb, the word “model” has become the symbol for 
the latest financial crisis because of the use, or more precisely, the misuse of such 
simulations to price unregistered, illiquid securities such as sub-prime mortgage 
backed securities and derivatives of such securities. The anecdotal cases where er- 
rant models have led to mischief are many and are not limited to the world of fi- 
nance alone. 

The Trouble with Models 

The problem is not with models themselves. The trouble happens when they are 
(a) improperly constructed and then (b) deliberately misapplied by individuals work- 
ing in the financial markets. 

In the physical sciences, models can be very usefully employed to help analysts 
understand complex systems such as disease, buildings and aircraft. These models 
tend to use observable data as inputs, can be scientifically validated and are codified 
in a manner that is transparent to all involved in the process. Models used in the 
physical world share one thing in common that financial models do not: they are 
connected to and are confirmed or refuted by the physical world they describe. 

Financial models, on the other hand, are all intellectual abstractions designed to 
manipulate arbitrarily chosen, human invented concepts. The chief reason for this 
digression from the objective use of models observed in the physical sciences is the 
injection of economics into the world of finance. Whereas financial models were once 
merely arithmetic expressions of expected cash flows, today in the world of financial 
economics, models have become vehicles for rampant speculation and outright 
fraud . 2 

In the world of finance, modeling has been an important part of the decision-mak- 
ing toolkit of executives and analysts for centuries, helping them to understand the 
various components in a company or a market and thereby adjust to take advantage 
of the circumstances. These decision analysis models seek to measure and report on 
key indicators of actual performance and confirm the position of the entity with re- 
spect to its’ competitive environment. For instance, the arithmetic calculation of 
cash flows adheres to the scientific method of structures and dynamics, and is the 
foundation of modern finance as embodied by the great theorists such as Benjamin 
Graham and David Dodd. 


1 Mr. Whalen is a co-founder of Institutional Risk Analytics, a Los Angeles unit of Lord, 
Whalen LLC that publishes risk ratings and provides customized financial analysis and valu- 
ation tools. 

2 See “New Hope for Financial Economics: Interview with Bill Janeway,” The Institutional 
Risk Analyst, November 17, 2008. 
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At our firm, we employ a “measure and report” model called The IRA Bank Mon- 
itor to survey and stress test all FDIC insured banks each quarter. By bench-mark- 
ing the performance of banks with a consistent set of tests, we are able to not only 
characterize the relative safety and soundness of each institution, but can drawn 
reasonable inferences about the bank’s future performance. 

But when the world of finance marries the world of outcome driven economics — 
the world of “what if’ and “I want” — models cease to be mechanistic tools for vali- 
dating current outcomes with hard data and assessing a reasonable range of pos- 
sible future events. Instead models become enablers for speculation, for the use of 
skillful canards and legal subterfuge that ultimately cheat investors and cause hun- 
dreds of billions of dollars in losses to private investors and insured depository insti- 
tutions. 

Take the world of mortgage backed securities or MBS. For decades the investment 
community had been using relatively simple models to predict the cash flow of MBS 
in various interest rate scenarios. These predictions have been relatively simple and 
are validated against the monthly mortgage servicer data available to the analyst 
community. The MBS securitization process was simple as well. A bank would sell 
conforming loans to GNMA and FNMA, and sell inferior collateral to a handful of 
investment banks on Wall Street to turn in the loans into private MBS issues. 

At the beginning of the 1990’s, however, Wall Street’s private MBS secret sauce 
escaped. A firm named Drexel, Burnham, Lambert went bankrupt and the bank- 
ruptcy court sold copies of Drexel’s structured finance software to anyone and every- 
one. It eventually wound up in the hands of the mortgage issuers themselves. These 
banks and non-banks naturally began to issue private MBS by themselves and dis- 
covered they could use the mathematics of modeling to grow their mortgage conduit 
businesses into massive cash flow machines. When brought to market, these private 
MBS were frequently under-collateralized and could therefore be described as a 
fraud. 

Wall Street, in turn, created even more complex modeling systems to squeeze even 
more profits from the original MBS template. The expanding bubble of financial in- 
novation caught the eye of policy-makers in the Congress, who then created political 
models envisioning the possibility that “innovation” could be used to make housing 
accessible to more Americans. 

Spurred on to chase the “policy outcome” of affordable housing, an entire range 
of deliberately opaque and highly leveraged financial instruments were born with 
the full support of Washington, the GSEs and the Congress. Their purpose now was 
to use the alchemy of financial modeling to create the appearance of mathematical 
safety out of dangerous toxic ingredients. Wall Street firms paid the major rating 
agencies to award “AAA” ratings to derivative assets that were ultimately based on 
sub-prime mortgage debt. And the stage was set for a future economic disaster. 

In the case of sub-prime toxic waste, the models became so complex that all trans- 
parency was lost. The dealers of unregulated, unregistered complex structured as- 
sets used proprietary models to price and sell deals, but since the “underlying” for 
these derivative securities was invisible, none of the investment or independent rat- 
ings community could model the security. There was no validation, no market dis- 
cipline. Buy Side customers were dependent upon the dealer who sold them the 
toxic waste for valuation. The dealers that controlled the model often time would 
not even make a market in the security. 

Clearly we have now many examples where a model or the pretense of a model 
was used as a vehicle for creating risk and hiding it. More important, however, is 
the role of financial models for creating opportunities for deliberate acts of securities 
fraud. These acts of fraud have caused hundreds of billions of dollars in losses to 
depository institutions and investors. 

Whether you talk about toxic mortgage assets or credit default swaps, the one 
common element that the misuse of models seems to contain is a lack of a visible 
underlying market against which to judge or “mark” the model. Indeed, the use of 
models in a subjective context seems to include the simulation of a nonexistent mar- 
ket as the primary role for the financial model. 

In single-name credit default swaps or “CDS” for example, there is often insuffi- 
cient trading in the supposed underlying corporate debt security to provide true 
price discovery. In the case of CDS on complex structured assets, there is no under- 
lying market to observe at all. The subjective model becomes tbe market in terms 
of pricing the security. 

In the spring of 2007, however, the fantasy land consensus that allowed people 
to believe that a model is a market came undone. We have been dealing with the 
consequences of the decisions that originally built the house of cards since that time. 
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An Objective Basis for Finance and Regulation 

The term “model” as it applies to finance can be a simulation of reality in terms 
of predicting future financial outcomes. The author Nassim Taleb, who is appearing 
at this hearing, says the term “VaR” or value at risk describes a statistical estimate 
of “the expected maximum loss (or worst loss) over a target horizon within a given 
confidence interval .” 3 

VaR models and similar statistical methods pretend to estimate the largest pos- 
sible loss that an investor might experience over a given period of time to a given 
degree of certainty. The use of VaR type models, including the version embedded 
in the Basel II agreement, involves a number of assumptions about risk and out- 
comes that are speculative. More important, the widespread use of these statistical 
models for risk management suggest that financial institutions are subject to occa- 
sional “Black Swans” in the form of risk events that cannot be anticipated. 

We take a different view. We don’t actually believe there is such a thing as a 
“Black Swan.” Our observations tell us that a more likely explanation is that lead- 
ers in finance and politics simply made the mistake of, again, believing in what 
were in fact flawed models and blinded themselves to what should have been plainly 
calculable innovation risks destined to be unsustainable. Or worse, our leaders in 
Washington and on Wall Street decided to be short sighted and not care about the 
inevitable debacle. 

We suggest that going forward our national interest needs to demand a higher 
standard of tangible proof from “outcome designers” of public policies. If financial 
markets and the models used to describe them are limited to those instruments that 
can be verified objectively, then we no longer need to fear from the ravages of Black 
Swans or systemic risk. The source of systemic risk in the financial markets is fear 
born from the complexity of opaque securities for which there is no underlying basis. 
The pretext for issuing these ersatz securities depends on subjectivity injected into 
a flawed model. 

If we accept that the sudden change in market conditions or the “Black Swan” 
event that Taleb and other theorists have so elegantly described arises from a 
breakdown in prudential regulation and basic common sense, and not from some un- 
knowable market mechanism, then we no longer need to fear surprises or systemic 
risk. We need to simply ensure that all of the financial instruments in our market- 
place have an objective basis, including a visible, cash basis market that is visible 
to all market participants. If investors cannot price a security without reference to 
subjective models, then the security should be banned from the U.S. markets as a 
matter of law and regulation. To do otherwise is to adopt deception as the public 
policy goal of the U.S. when it comes to financial markets regulation. 

As Graham and Dodd wrote nearly a century ago, the more speculative the inputs 
the less the analysis matters. Models only have real value to society when their 
workings are disciplined by the real world. When investors, legislators and regu- 
lators all mistook models for markets, and even accepted such speculations as a 
basis for regulating banks and governing over-the-counter or OTC markets for all 
types of securities, we as a nation were gambling with our patrimony. If the Com- 
mittee and the Congress want to bring an end to the financial crisis, we must de- 
mand higher standards from our citizens who work in and regulate our financial 
markets. 

As we discussed in a commentary last month, “Systemic Risk: Is it Black Swans 
or Market Innovations?,” published in The Institutional Risk Analyst, “were the fail- 
ures of Bear Stearns, Lehman Brothers, Washington Mutual or the other “rare” 
events really anomalous? Or are we just making excuses for our collective failure 
to identify and manage risk? A copy of our commentary follows this testimony. I 
look forward to your questions. 


3 See Taleb, Nassim, "Against Val ue-at-Risk: Nassim Taleb Replies to Philippe Jorion,” 1997. 
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Systemic Risk: Is it Black Swans 
or Market Innovations? 

August 18, 2009 

“Whatever you think you know about the distribution changes the distribution.” 

Alex Pollock 

American Enterprise Institute 

In this week’s issue of The IRA, our friend and colleague Richard Alford, a former 
Fed of New York economist, and IRA founders Dennis Santiago and Chris Whalen, 
ask us whether we really see Black Swans in market crises or our own expectations. 
Of note, we will release our preliminary Q2 Banking Stress Index ratings on Mon- 
day, August 24, 2009. As with Ql, these figures represent about 90 percent of all 
FDIC insured depositories, but exclude the largest money center banks (aka the 
“Stress Test Nineteen”), thus providing a look at the state of the regional and commu- 
nity banks as of the quarter ended June 30, 2009. Click here to register for The Insti- 
tutional Risk Analyst. 

Many popular explanations of recent financial crises cite “Black Swan” events; ex- 
treme, unexpected, “surprise” price movements, as the causes of the calamity. How- 
ever, in looking at our crisis wracked markets, we might consider that the Black 
Swan hypothesis doesn’t fit the facts as well an alternative explanation: namely that 
the speculative outburst of financial innovation and the artificially low, short-run 
interest rate environment pursued by the Federal Open Market Committee, com- 
bined to change the underlying distribution of potential price changes. This shift in 
the composition of the distribution made likely outcomes that previously seemed im- 
possible or remote. This shift in possible outcomes, in turn, generated surprise in 
the markets and arguably led to the emergence of “systemic risk” as a metaphor 
to explain these apparent “anomalies.” 

But were the failures of Bear Stearns, Lehman Brothers, Washington Mutual or 
the other “rare” events really anomalous? Or are we just making excuses for our 
collective failure to identify and manage risk? 

The choice of which hypothesis to ultimately accept in developing the narrative 
description of the causation of the financial crisis has strategic implications for un- 
derstanding as well as reducing the likelihood of future crisis, including the effect 
on the safety and soundness of financial institutions. To us, the hard work is not 
trying to specifically limit the range of possibilities with artificial assumptions, but 
to model risk when you must assume as a hard rule, like the rules which govern 
the physical sciences, that the event distribution is in constant flux. 

If we as financial and risk professional are serious in claims to model risk 
proactively, then change, not static assumptions, must be the rule in terms of the 
possible outcomes. Or “paranoid and nimble” in practical terms. After all, these 
modeling exercises ultimately inform and support risk assumptions for decisions 
that are used in value-at-risk (VaR) assessments for investors and for capital ade- 
quacy bench-marking for financial institutions. 

Even before the arrival of Benoit Mandelbrot in the 1960s, researchers had ob- 
served that distributions of price changes in various markets were not normally dis- 
tributed. The observed distributions of price changes had fatter tails than the nor- 
mal distribution. Nassim Nicolas Taleb, author of The Black Swan and Fooled by 
Randomness, and others have dubbed significantly larger extreme price moves than 
those predicted by a normal distribution as “Black Swans.” Indeed, Taleb and others 
have linked Black Swan price change events to the recent financial crisis, sug- 
gesting in effect that we all collectively misunderstood on which side of the distribu- 
tion of possible risk outcomes we stood. 

The argument is as follows: Current risk management and derivative pricing re- 
gimes are based upon normal distributions. Price movements in the recent financial 
crises were unpredictable/low probability events that were also greater than pre- 
dicted by normal distribution models. Hence our collective failure to anticipate 
Black Swan events is “responsible” for the recent crises as mis-specified risk man- 
agement models failed due to fatter than normal tails. 

The alternative explanation, however, links the extreme price movements not to 
aberrations with respect to a stable, observable mean, but instead to the activation 
of alternate stable means as a result of jumping discontinuously through tipping 
points — much in the same way particles jump quantum levels in energy states when 
subjected to the cumulative effects of energy being added to or removed from their 
environments. These tipping points are as predictable as the annual migrations of 
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ducks. Swans, alas, rarely migrate, preferring to stay in their summer feeding 
grounds until the water freezes, then move only far enough to find open water. 
Sound familiar? 

Force feed a system with enough creative energy via permissive public policies 
and the resulting herd behaviors, and the system will change to align around these 
new norms, thereby erasing the advantages of the innovators and creating unfore- 
seen hazards. “Advances” such as OTC derivatives and complex structured assets, 
and very accommodating Fed interest rate policy, resulted in unprecedented lever- 
age and maturity mismatches by institutions and in markets that are the perfect 
quantum fuel to brew such change. 

While the exact timing of each tipping point and magnitude of the crises remains 
somewhat inexact, the waves of change and the ultimate crisis borne shift are 
broadly predictable. The probabilities attached to extreme price moves are calculable 
as the cost of deleveraging an accumulation of innovation risk that must be shed 
as the system realigns. The “Black Swan” approach assumes a stable distribution 
of price changes with fatter than “normal” tails. The alternative posits that the dis- 
tribution of possible price changes was altered by innovation and the low cost of le- 
verage. It also posits that the new distributions allowed, indeed require, more ex- 
treme price movements. Two examples will illustrate the alternative hypothesis. 

Once upon a time, the convertible bond market was relatively quiet. The buy side 
was dominated by real money (unleveraged) players who sought the safety of bonds, 
but were willing to give up some return for some upside risk (the embedded equity 
call option). 

More recently the market has been dominated by leveraged hedge funds doing 
convertible bond arbitrage. They bought the bonds, hedging away the various risks. 
In response to the advent of the arbitrageurs, the spread between otherwise similar 
conventional and convertible bonds moved to more accurately reflect the value of the 
embedded option and became less volatile. 

When the financial crises hit, however, arbitrageurs were forced to liquidate their 
positions as losses mounted and it became difficult to fund the leveraged positions. 
Prices for convertible bonds declined and for a period were below prices for similar 
conventional bonds — something that had been both unheard of and considered im- 
possible as the value of an option cannot be negative. 

Was this a Black Swan type event, or had the market for convertible bonds and 
the underlying distribution of price changes, been altered? The mean spread be- 
tween otherwise similar conventional and convertible bonds had changed. The vola- 
tility of the spread had changed. Forced sales and the public perception of possible 
future forced sales generated unprecedented behavior of the heretofore stable 
spread. The emergence and then dominance of leveraged arbitrage positions altered 
the market in fundamental ways. What had not been possible had become possible. 

Now consider bank exposures to commercial real estate. Numerous financial insti- 
tutions, hedge funds (e.g., at Bear Stearns), sellers of CDS protection (e.g., AIG) and 
banks (many of them foreign as reflected in the Fed swap lines with foreign central 
banks) suffered grievous losses when the real estate bubble popped. Much of these 
losses remain as yet unrealized. 

As investors and regulators demanded asset-write downs and loss realization, 
many of these institution expressed dismay. They had stressed tested their port- 
folios, the large banks complained, often with the support of regulators. The large 
banks thought their geographically diversified portfolios of MBSs immunize them 
from falls in real estate prices as the US had experienced regional, but never (except 
for the 1930s) nationwide declines in housing prices. These sophisticated banks in- 
corporated that assumption into their stress test even as they and the securitization 
process were nationalizing — that is, changing — the previously regional and local 
mortgage markets. 

Was the nationwide decline in housing prices an unpredictable Black Swan event 
or the foreseeable result of lower lending standards, a supportive interest rate envi- 
ronment, and financial innovation the led to the temporary nationalization of the 
mortgage market? Risk management regimes failed and banks have been left with 
unrealized losses that still threaten the solvency of the entire system in Q3 2009. 

However useful or necessary “normal” statistical measures such as VaR might be, 
it will not be sufficient to insulate institutions or the system from risk arising from 
rapidly evolving market structures and practices. Furthermore, insofar as models 
such as VaR, which are now enshrined in the bank regulatory matrix via Basel II, 
were the binding constraint on risk taking, it acted perversely, allowing ever greater 
leverage as leveraged trading acted to reduce measured volatility! Remember, the 
convertible bond market at first looked placid as a lake as leverage grew — but then 
imploded in a way few thought possible. Is this a Black Swan event or a failure of 
the stated objectives of risk management and prudential oversight? 
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We all know that risk management systems based solely on analysis of past price 
moves will at some point fall if financial markets continue to change. The problem 
with current risk management systems cannot be fixed by fiddling with VaR or 
other statical models. Risk management regimes must incorporate judgments about 
the evolution of the underlying markets, distribution of possible price changes and 
other dynamic sources of risk. 

Indeed, as we discussed last week (“Are You Ready for the Next Bank Stress 
Tests”), this is precisely why IRA employs quarterly surveys of bank stress tests to 
benchmark the US banking industry. Think of the banking industry as a school of 
fish, moving in generally the same direction, but not uniformly or even consistently. 
There is enormous variation in the past of each member of the school, even though 
from a distance the group seems to move in unison. 

Stepping back from the narrow confines of finance for a moment, consider that 
the most dramatic changes in the world are arguably attributable to asymmetric 
confluences of energy changing the direction of human history. It’s happened over 
and over again. The danger has and always will be the immutable law of unin- 
tended consequences, which always comes back to bite the arrogant few who believe 
they can control the future outcome. And it is always the many of us who pay the 
price for these reckless leaps of faith. 

If the recent financial crises were truly highly infrequent random events, then any 
set of policies that can continuously prevent their reoccurrence seemingly will be 
very expensive in terms of idle capital and presumably less efficient markets re- 
quired to avoid them. If, on the other hand, the crisis was the result of financial 
innovation and the ability to get leveraged cheaply, then society need not continu- 
ously bare all the costs associated with preventing market events like the bursting 
of asset bubbles. 

Policy-makers would like everyone to believe that the recent crises were random 
unpredictable Black Swan events. How can they be blamed for failing to anticipate 
a low probability, random, and unpredictable event? If on the other hand, the crises 
had observable antecedents, e.g., increased use of leverage, maturity mismatches, 
near zero default rates, and spikes in housing price to rental rates and housing 
price to income ratios, then one must ask: why policy-makers did not connect the 
dots, attach significant higher than normal probabilities to the occurrence of severe 
financial disturbances, and fashion policies accordingly? Ultimately, that is a ques- 
tion that Ben Bernanke and the rest of the federal financial regulatory community 
still have yet to answer. 

Questions? Comments? info@institutionalriskanalytics.com 
Biography for Christopher Whalen 

Christopher is co-founder of Institutional Risk Analytics, the Los Angeles based 
provider of risk management tools and consulting services for auditors, regulators 
and financial professionals. Christopher leads IRA’s risk advisory practice and 
consults for global companies on a variety of financial and regulatory issues. He is 
a Fellow of the Networks Financial Institute at Indiana State University. Chris- 
topher volunteers as a regional director of Professional Risk Managers International 
Association (www.prmia.org) and is a board adviser to I-OnAsia Limited 
( www.ionasia.com.hk ), a global business security and risk consultancy based in 
Hong Kong. Christopher currently edits The Institutional Risk Analyst, a weekly 
news report and commentary on significant developments in and around the global 
financial markets. Christopher has testified before the Congress and the SEC on a 
variety of issues and contributes articles and commentaries to publications such as 
The International Economy, American Banker and The Big Picture. 

Chairman Miller. Thank you. 

Dr. Colander. 

STATEMENT OF DR. DAVID COLANDER, CHRISTIAN A. JOHN- 
SON DISTINGUISHED PROFESSOR OF ECONOMICS, 
MIDDLEBURY COLLEGE 

Dr. Colander. Mr. Chairman, thanks for the opportunity to tes- 
tify. I am Dave Colander, the Christian A. Johnson Distinguished 
Professor of Economics at Middlebury College. I was invited here 
because I was one of the authors of the Dahlem Report in which 
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we chided the economics profession for its failure to warn society 
about the impending financial crisis. 

Some non-economists have blamed the financial crisis on econo- 
mists’ highly technical models. My argument is that the problem 
isn’t the models, the problem is the way the economic models are 
used, and I think a number of the other panelists have made that 
point. Where I am going to lead or go with that is that the issue 
goes much deeper than just with VaR and the various models you 
are looking at, and it goes very much to the general arguments 
about science and technology and the way in which economists ap- 
proach problems, and I think, you know, Mr. Whalen had it di- 
rectly right: we live in the world of supposition. Why? Because that 
is what our incentives are. We write articles. We advance through 
writing articles, we don’t advance by designing something positive. 
If we are working for a business, we do, but within academics it 
is very much directed towards, you know, sort of what can we pub- 
lish, and so I think Value-at-Risk models are part of a much broad- 
er economic problem, you know, sort of in terms of what economists 
accept and how they go about doing what they are doing. 

An example I want to give is really about macroeconomics, you 
know, sort of in the dominant model in macroeconomics, which is 
the dynamic stochastic general equilibrium (DSGE) model, which is 
a big model designed very much along the same lines about effi- 
cient markets. It sort of took efficient markets and said, what if we 
had efficient markets in the entire economy? To get that model, you 
have to assume there is one individual, because we can’t solve it 
unless there is only one individual. We have to assume that person 
is globally rational, understands everything and he has complete 
knowledge in looking into the infinite future, and then we can actu- 
ally solve it for a very small case. 

By definition, this model rules out strategic coordination prob- 
lems. What would happen if somebody else did something else? 
That is obviously the likely cause of the recent crisis, but it was 
simply assumed away in the macroeconomic model and that macro- 
economic model has been dominant for the last 30 years and has 
been funded by NSF, the research, you need to be looking into that. 

If the DSGE model had been seen as an aid to common sense, 
it could have been a useful model. It improved some of the prob- 
lems that some earlier models had. But for a variety of sociological 
reasons that I don’t have time to go into here, a majority of macro- 
economists started believing the DSGE model was useful, not just 
as an aid to our understanding but as the model of the macro- 
economy. As that DSGE model became dominant, really important 
research on the whole set of broader non-linear and complex dy- 
namic models that would have really served some foundation for 
thinking about these issues just wasn’t done. It just wasn’t allowed. 
You couldn’t get anything published on it in the main macro jour- 
nals. 

Similar developments occurred with the efficient market finance 
models, which made assumptions very similar to the DSGE model. 
And so, again, at first these served a useful purpose. They led to 
technological advances in risk management and financial markets. 
But as happened in macro, the users of these financial models for- 
got that the models provide, at best, half-truths. They stopped 
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using models with common sense and judgment. What that means 
is that warning labels should be put on models, and that should 
be in bold print, ‘these models are based on assumptions that do 
not fit the real world and thus these models should be not relied 
on very heavily.’ Those warning labels haven’t been there. 

How did something so stupid like this happen in economics? It 
didn’t happen because economists are stupid, and I appreciate the 
people before who said we are not. We are very bright. It happened 
because of incentives within the economics profession and those in- 
centives lead researchers to dot i’s and cross t’s of existing models. 
It is a lot easier to do that than to design a whole new model that 
nobody else, a peer, can really review. So they don’t explore the 
wide range of alternative models, and they don’t focus their re- 
search on interpreting and seeing that models are used in policy 
in a common sense fashion. 

So let me conclude with just two brief suggestions which relate 
to issues under the jurisdiction of this committee that might de- 
crease the probability of such events happening in the future, and 
these are far off but it has to do with, you know, sort of the incen- 
tives for economists. The first is a proposal that might add some 
common sense check on models. Such a check is needed because 
currently there is a nature of the internal to the sub-field peer re- 
view system, that works within NSF and within the system, that 
allows for what can only be called an incestuous mutual reinforce- 
ment of researchers’ views with no common sense filter on those 
views. My proposal is to include a wider range of peers in the re- 
viewing process for the National Science Foundation grants in the 
social sciences. For example, physicists, mathematicians, statisti- 
cians and even business and government representatives could 
serve on reviewing those, and it would serve as a useful common 
sense check, you know, about what is going on. 

The second is a proposal to increase the number of researchers 
trained in interpreting models, rather than developing models, by 
providing research grants to do precisely that. In a sense, what I 
am suggesting is an applied science division of the National Science 
Foundation, a social science component. This division would fund 
work on the usefulness of models and would be responsible for add- 
ing the warning labels that should have been attached to those 
models. 

The applied research would not be highly technical and would in- 
volve a quite different set of skills than the standard scientific re- 
search requires. It would require researchers to have an intricate 
knowledge — consumer’s knowledge of the theory, but not a pro- 
ducer’s knowledge of that theory. In addition, it would require a 
knowledge of institutions, methodology, previous literature and a 
sensibility of how the system works. These are all skills that are 
not taught in graduate economics today, but they are skills that 
underlie judgment and common sense. By providing NSF grants for 
this work, the NSF would encourage the development of a group 
of economists who specialize in interpreting models and applying 
models to the real world. The development of such a group would 
go a long way toward placing the necessary warning labels on mod- 
els. Thank you. 

[The prepared statement of Dr. Colander follows:] 
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Prepared Statement of David Colander 

Mr. Chairman and Members of the Committee: I thank you for the opportunity 
to testify. My name is David Colander. I am the Christian A. Johnson Distinguished 
Professor of Economics at Middlebury College. I have written or edited over forty 
books, including a top-selling principles of economics textbook, and 150 articles on 
various aspects of economics. I was invited to speak because I was one of the au- 
thors of the Dahlem Report in which we chided the economics profession for its fail- 
ure to warn society about the impending financial crisis, and I have been asked to 
expand on some of the themes that we discussed in that report. (I attach that report 
as an appendix to this testimony.) 

Introduction 

One year ago, almost to the day, the U.S. economy had a financial heart attack, 
from which it is still recovering. That heart attack, like all heart attacks, was a 
shock, and it has caused much discussion about who is to blame, and how can we 
avoid such heart attacks in the future. In my view much of that discussion has been 
off point. To make an analogy to a physical heart attack, the U.S. had a heart at- 
tack because it is the equivalent of a 450-pound man with serious ailments too nu- 
merous to list, who is trying to live as if he were still a 20-year-old who can party 
24—7. It doesn’t take a rocket economist to know that that will likely lead to trouble. 
The questions I address in my testimony are: Why didn’t rocket economists recog- 
nize that, and warn society about it? And: What changes can be made to see that 
it doesn’t happen in the future? 

Some non-economists have blamed the financial heart attack on economist’s high- 
ly technical models. In my view the problem is not the models; the problem is the 
way economic models are used. All too often models are used in lieu of educated 
common sense, when in fact models should be used as an aid to educated common 
sense. When models replace common sense, they are a hindrance rather than a help. 

Modeling the Economy as a Complex System 

Using models within economics or within any other social science, is especially 
treacherous. That’s because social science involves a higher degree of complexity 
than the natural sciences. The reason why social science is so complex is that the 
basic unit in social science, which economists call agents, are strategic, whereas the 
basic unit of the natural sciences are not. Economics can be thought of the physics 
with strategic atoms, who keep trying to foil any efforts to understand them and 
bring them under control. Strategic agents complicate modeling enormously; they 
make it impossible to have a perfect model since they increase the number of cal- 
culations one would have to make in order to solve the model beyond the calcula- 
tions the fastest computer one can hypothesize could process in a finite amount of 
time. 

Put simply, the formal study of complex systems is really, really, hard. Inevitably, 
complex systems exhibit path dependence, nested systems, multiple speed variables, 
sensitive dependence on initial conditions, and other non-linear dynamical prop- 
erties. This means that at any moment in time, right when you thought you had 
a result, all hell can break loose. Formally studying complex systems requires rig- 
orous training in the cutting edge of mathematics and statistics. It’s not for neo- 
phytes. 

This recognition that the economy is complex is not a new discovery. Earlier 
economists, such as John Stuart Mill, recognized the economy’s complexity and were 
very modest in their claims about the usefulness of their models. They carefully pre- 
sented their models as aids to a broader informed common sense. They built this 
modesty into their policy advice and told policy-makers that the most we can expect 
from models is half-truths. To make sure that they did not claim too much for their 
scientific models, they divided the field of economics into two branches-one a sci- 
entific branch, which worked on formal models, and the other political economy, 
which was the branch of economics that addressed policy. Political economy was 
seen as an art which did not have the backing of science, but instead relied on the 
insights from models developed in the scientific branch supplemented by educated 
common sense to guide policy prescriptions. 

In the early 1900s that two-part division broke down, and economists became a 
bit less modest in their claims for models, and more aggressive in their application 
of models directly to policy questions. The two branches were merged, and the result 
was a tragedy for both the science of economics and for the applied policy branch 
of economics. 

It was a tragedy for the science of economics because it led economists away from 
developing a wide variety of models that would creatively explore the extraor- 
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dinarily difficult questions that the complexity of the economy raised, questions for 
which new analytic and computational technology opened up new avenues of inves- 
tigation. 1 Instead, the economics profession spent much of its time dotting i’s and 
crossing t’s on what was called a Walrasian general equilibrium model which was 
more analytically tractable. As opposed to viewing the supply/demand model and its 
macroeconomic counterpart, the Walrasian general equilibrium model, as interesting 
models relevant for a few limited phenomena, but at best a stepping stone for a for- 
mal understanding of the economy, it enshrined both models, and acted as if it ex- 
plained everything. Complexities were just assumed away not because it made sense 
to assume them away, but for tractability reasons. The result was a set of models 
that would not even pass a perfunctory common sense smell test being studied ad 
nauseam. 

Initially macroeconomics stayed separate from this broader unitary approach, and 
relied on a set of rough and ready models that had little scientific foundation. But 
in the 1980s, macroeconomics and finance fell into this “single model” approach. As 
that happened it caused economists to lose sight of the larger lesson that complexity 
conveys — that models in a complex system can be expected to continually break 
down. This adoption by macroeconomists of a single-model approach is one of the 
reasons why the economics profession failed to warn society about the financial cri- 
sis, and some parts of the profession assured society that such a crisis could not 
happen. Because they focused on that single model, economists simply did not study 
and plan for the inevitable breakdown of systems that one would expect in a com- 
plex system, because they had become so enamored with their model that they for- 
got to use it with common sense judgment. 

Models and Macroeconomics 

Let me be a bit more specific. The dominant model in macroeconomics is the dy- 
namic stochastic general equilibrium (DSGE) model. This is a model that assumes 
there is a single globally rational representative agent with complete knowledge 
who is maximizing over the infinite future. In this model, by definition, there can 
be no strategic coordination problem — the most likely cause of the recent crisis — 
such problems are simply assumed away. Yet, this model has been the central focus 
of macro economists’ research for the last thirty years. 

Had the DSGE model been seen as an aid to common sense, it could have been 
a useful model. When early versions of this model first developed back in the early 
1980s, it served the useful purpose of getting some inter-temporal issues straight 
that earlier macroeconomic models had screwed up. But then, for a variety of socio- 
logical reasons that I don’t have time to go into here, a majority of macroeconomists 
started believing that the DSGE model was useful not just as an aid to our under- 
standing, but as the model of the macroeconomy. That doesn’t say much for the com- 
mon sense of rocket economists. As the DSGE model became dominant, important 
research on broader non-linear dynamic models of the economy that would have 
been more helpful in understanding how an economy would be likely to crash, and 
what government might do when faced with a crash, was not done. 2 

Similar developments occurred with efficient market finance models, which make 
similar assumptions to DSGE models. When efficient market models first developed, 
they were useful; they led to technological advances in risk management and finan- 
cial markets. But, as happened with macro, the users of these financial models for- 
got that models provide at best half truths; they stopped using models with common 
sense and judgment. The modelers knew that there was uncertainty and risk in 
these markets that when far beyond the risk assumed in the models. Simplification 
is the nature of modeling. But simplification means the models cannot be used di- 
rectly, but must be used judgment and common sense, with a knowledge of the limi- 
tations of use that the simplifications require. Unfortunately, the warning labels on 
the models that should have been there in bold print — these models are based 
on assumptions that do not fit the real world, and thus the models should 
not be relied on too heavily — were not there. They should have been, which is 


1 Some approaches working outside this Walrasian general equilibrium framework that I see 
as promising includes approaches using adaptive network analysis, agent based modeling, ran- 
dom graph theory, ultrametrics, combinatorial stochastic processes, co-integrated vector auto-re- 
gression, and the general study of non-linear dynamic models. 

2 Among well known economists, Robert Solow stands out in having warned about the use of 
DSGE models for policy. (See Solow, in Colander, 2007, pg. 235.) He called them “rhetorical 
swindles.” Other economists, such as Post Keynesians, and economic methodologists also warned 
about the use of these models. For a discussion of alternative approaches, see Colander, ed. 
(2007). So alternative approaches were being considered, and concern about the model was 
aired, but those voices were lost in the enthusiasm most of the macroeconomics community 
showed for these models. 
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why in the Dahlem Report we suggested that economic researchers who develop 
these models be subject to a code of ethics that requires them to warn society when 
economic models are being used for purposes for which they were not designed. 

How did something so stupid happen in economics? It did not happen because 
economists are stupid; they are very bright. It happened because of incentives in 
the academic profession to advance lead researchers to dot i’s and cross t’s of exist- 
ing models, rather than to explore a wide range of alternative models, or to focus 
their research on interpreting and seeing that models are used in policy with com- 
mon sense. Common sense does not advance one very far within the economics pro- 
fession. The over-reliance on a single model used without judgment is a serious 
problem that is built into the institutional structure of academia that produces eco- 
nomic researchers. That system trains show dogs, when what we need are hunting 
dogs. 

The incorrect training starts in graduate school, where in their core courses stu- 
dents are primarily trained in analytic techniques useful for developing models, but 
not in how to use models creatively, or in how to use models with judgment to arrive 
at policy conclusions. For the most part policy issues are not even discussed in the 
entire core macroeconomics course. As students at a top graduate school said, “Mon- 
etary and fiscal policy are not abstract enough to be a question that would be an- 
swered in a macro course” and “We never talked about monetary or fiscal policy, 
although it might have been slipped in as a variable in one particular model.” (Col- 
ander, 2007, pg. 169). 

Suggestions 

Let me conclude with a brief discussion of two suggestions, which relate to issues 
under the jurisdiction of this committee, that might decrease the probability of such 
events happening in the future. 

Include a wider range of peers in peer review 

The first is a proposal that might help add a common sense check on models. Such 
a check is needed because, currently, the nature of internal-to-the-subfield peer re- 
view allows for an almost incestuous mutual reinforcement of researcher’s views 
with no common sense filter on those views. The proposal is to include a wider 
range of peers in the reviewing process of NSF grants in the social sciences. For 
example, physicists, mathematician, statisticians, and even business and govern- 
mental representatives, could serve, along with economists, on reviewing committees 
for economics proposals. Such a broader peer review process would likely both en- 
courage research on much wider range of models and would also encourage more 
creative work. 

Increase the number of researchers trained to interpret models 

The second is a proposal to increase the number of researchers trained in inter- 
preting models rather than developing models by providing research grants to do 
that. In a sense, what I am suggesting is an applied science division of the National 
Science Foundation’s social science component. This division would fund work on the 
usefulness of models, and would be responsible for adding the warning labels that 
should have been attached to the models. 

This applied research would not be highly technical and would involve a quite dif- 
ferent set of skills than the standard scientific research would require. It would re- 
quire researchers who had an intricate consumer’s knowledge of theory but not a 
producer’s knowledge. In addition it would require a knowledge of institutions, 
methodology, previous literature, and a sensibility about how the system works. 
These are all skills that are currently not taught in graduate economics programs, 
but they are the skills that underlie judgment and common sense. By providing NSF 
grants for this work, the NSF would encourage the development of a group of econo- 
mists who specialized in interpreting models and applying models to the real world. 
The development of such a group would go a long way toward placing the necessary 
warning labels on models, and make it less likely that fiascoes like a financial crisis 
would happen again. 
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1. Introduction 

The global financial crisis has revealed the need to rethink fundamentally how 
financial systems are regulated. It has also made clear a systemic failure of the eco- 
nomics profession. Over the past three decades, economists have largely developed 
and come to rely on models that disregard key factors — including heterogeneity of 
decision rules, revisions of forecasting strategies, and changes in the social context — 
that drive outcomes in asset and other markets. It is obvious, even to the casual 
observer that these models fail to account for the actual evolution of the real-world 
economy. Moreover, the current academic agenda has largely crowded out research 
on the inherent causes of financial crises. There has also been little exploration of 
early indicators of system crisis and potential ways to prevent this malady from de- 
veloping. In fact, if one browses through the academic macroeconomics and finance 
literature, “systemic crisis” appears like an otherworldly event that is absent from 
economic models. Most models, by design, offer no immediate handle on how to 
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think about or deal with this recurring phenomenon. 3 In our hour of greatest need, 
societies around the world are left to grope in the dark without a theory. That, to 
us, is a systemic failure of the economics profession. 

The implicit view behind standard equilibrium models is that markets and econo- 
mies are inherently stable and that they only temporarily get off track. The majority 
of economists thus failed to warn policy-makers about the threatening system crisis 
and ignored the work of those who did. Ironically, as the crisis has unfolded, econo- 
mists have had no choice but to abandon their standard models and to produce 
hand-waving common sense remedies. Common sense advice, although useful, is a 
poor substitute for an underlying model that can provide much-needed guidance for 
developing policy and regulation. It is not enough to put the existing model to one 
side, observing that one needs, “exceptional measures for exceptional times.” What 
we need are models capable of envisaging such “exceptional times.” 

The confinement of macroeconomics to models of stable states that are perturbed 
by limited external shocks and that neglect the intrinsic recurrent boom-and-bust 
dynamics of our economic system is remarkable. After all, worldwide financial and 
economic crises are hardly new and they have had a tremendous impact beyond the 
immediate economic consequences of mass unemployment and hyper inflation. This 
is even more surprising, given the long academic legacy of earlier economists’ study 
of crisis phenomena, which can be found in the work of Walter Bagehot (1873), Axel 
Leijonhuvfud (2000), Charles Kindleberger (1989), and Hyman Minsky (1986), to 
name a few prominent examples. This tradition, however, has been neglected and 
even suppressed. 

The most recent literature provides us with examples of blindness against the up- 
coming storm that seem odd in retrospect. For example, in their analysis of the risk 
management implications of CDOs, Krahnen (2005) and Krahnen and Wilde (2006) 
mention the possibility of an increase of ‘systemic risk.’ But, they conclude that this 
aspect should not be the concern of the banks engaged in tbe CDO market, because 
it is the governments’ responsibility to provide costless insurance against a system- 
wide crash. We do not share this view. On the more theoretical side, a recent and 
prominent strand of literature essentially argues that consumers and investors are 
too risk averse because of their memory of the (improbable) event of the Great De- 
pression (e.g., Cogley and Sargent, 2008). Much of the motivation for economics as 
an academic discipline stems from the desire to explain phenomena like unemploy- 
ment, boom and bust cycles, and financial crises, but dominant theoretical models 
exclude many of the aspects of the economy that will likely lead to a crisis. Con- 
fining theoretical models to ‘normal’ times without consideration of such defects 
might seem contradictory to the focus that the average taxpayer would expect of the 
scientists on his payroll. 

This failure has deep methodological roots. The often heard definition of econom- 
ics — that it is concerned with the ‘allocation of scarce resources’ — is short-sighted 
and misleading. It reduces economics to the study of optimal decisions in well-speci- 
fied choice problems. Such research generally loses track of the inherent dynamics 
of economic systems and the instability that accompanies its complex dynamics. 
Without an adequate understanding of these processes, one is likely to miss the 
major factors that influence the economic sphere of our societies. This insufficient 
definition of economics often leads researchers to disregard questions about the co- 
ordination of actors and the possibility of coordination failures. Indeed, analysis of 
these issues would require a different type of mathematics than that which is gen- 
erally used now by many prominent economic models. 

Many of the financial economists who developed the theoretical models upon 
which the modern financial structure is built were well aware of the strong and 
highly unrealistic restrictions imposed on their models to assure stability. Yet, fi- 
nancial economists gave little warning to the public about the fragility of their mod- 
els, 4 even as they saw individuals and businesses build a financial system based on 
their work. There are a number of possible explanations for this failure to warn the 
public. One is a “lack of understanding” explanation — the researchers did not know 
the models were fragile. We find this explanation highly unlikely; financial engi- 
neers are extremely bright, and it is almost inconceivable that sucb bright individ- 


3 Reinhart and Rogoff (2008) argue that the current financial crisis differs little from a long 
chain of similar crises in developed and developing countries. We certainly share their view. The 
problem is that the received body of models in macro finance to which the above authors have 
prominently contributed provides no room whatsoever for such recurrent boom and bust cycles. 
The literature has, therefore, been a major source of the illusory ‘this time it is different’ view 
that the authors themselves criticize. 

4 Indeed, few researchers explored the consequences of a breakdown of their assumptions, 
even though this was rather likely. 
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uals did not understand the limitations of the models. A second, more likely expla- 
nation, is that they did not consider it their job to warn the public. If that is the 
cause of their failure, we believe that it involves a misunderstanding of the role of 
the economist, and involves an ethical breakdown. In our view, economists, as with 
all scientists, have an ethical responsibility to communicate the limitations of their 
models and the potential misuses of their research. Currently, there is no ethical 
code for professional economic scientists. There should be one. 

In the following pages, we identify some major areas of concern in theory and ap- 
plied methodology and point out their connection to crisis phenomena. We also high- 
light some promising avenues of study that may provide guidance for future re- 
searchers. 

2. Models (or the Use of Models) as a Source of Risk 

The economic textbook models applied for allocation of scarce resources are pre- 
dominantly of the Robinson Crusoe (representative agent) type. Financial market 
models are obtained by letting Robinson manage his financial affairs as a sideline 
to his well-considered utility maximization over his (finite or infinite) expected life- 
span taking into account with correct probabilities all potential future happenings. 
This approach is mingled with insights from Walrasian general equilibrium theory, 
in particular the finding of the Arrrow-Debreu two-period model that all uncertainty 
can be eliminated if only there are enough contingent claims (i.e., appropriate deriv- 
ative instruments). This theoretical result (a theorem in an extremely stylized 
model) underlies the common belief that the introduction of new classes of deriva- 
tives can only be welfare increasing (a view obviously originally shared by former 
Fed Chairman Greenspan). It is worth emphasizing that this view is not an empiri- 
cally grounded belief but an opinion derived from a benchmark model that is much 
too abstract to be confronted with data. 

On the practical side, mathematical portfolio and risk management models have 
been the academic backbone of the tremendous increase of trading volume and di- 
versification of instruments in financial markets. Typically, new derivative products 
achieve market penetration only if a certain industry standard has been established 
for pricing and risk management of these products. Mostly, pricing principles are 
derived from a set of assumptions on an ‘appropriate’ process for the underlying 
asset, (i.e., the primary assets on which options or forwards are written) together 
with an equilibrium criterion such as arbitrage-free prices. With that mostly comes 
advice for hedging the inherent risk of a derivative position by balancing it with 
other assets that neutralize the risk exposure. The most prominent example is cer- 
tainly the development of a theory of option pricing by Black and Scholes that even- 
tually (in the eighties) could even be implemented on pocket calculators. Simulta- 
neously with Black-Scholes option pricing, the same principles led to the widespread 
introduction of new strategies under the heading of portfolio insurance and dynamic 
hedging that just tried to implement a theoretically risk-free portfolio composed of 
both assets and options and keep it risk-free by frequent rebalancing after changes 
of its input data (e.g., asset prices). For structured products for credit risk, the basic 
paradigm of derivative pricing — perfect replication — is not applicable so that one 
has to rely on a kind of rough-and-ready evaluation of these contracts on the base 
of historical data. Unfortunately, historical data were hardly available in most cases 
which meant that one had to rely on simulations with relatively arbitrary assump- 
tions on correlations between risks and default probabilities. This makes the theo- 
retical foundations of all these products highly questionable — the equivalent to 
building a building of cement of which you weren’t sure of the components. The dra- 
matic recent rise of the markets for structured products (most prominently 
collateralized debt obligations and credit default swaps — CDOs and CDSs) was 
made possible by development of such simulation-based pricing tools and the adop- 
tion of an industry-standard for these under the lead of rating agencies. Barry 
Eichengreen (2008) rightly points out that the “development of mathematical meth- 
ods designed to quantify and hedge risk encouraged commercial banks, investment 
banks and hedge funds to use more leverage” as if the very use of the mathematical 
methods diminished the underlying risk. He also notes that the models were esti- 
mated on data from periods of low volatility and thus could not deal with the arrival 
of major changes. Worse, it is our contention that such major changes are endemic 
to the economy and cannot be simply ignored. 

What are the flaws of the new unregulated financial markets which have 
emerged? As we have already pointed out in the introduction, the possibility of sys- 
temic risk has not been entirely ignored but it has been defined as lying outside 
the responsibility of market participants. In this way, moral hazard concerning sys- 
temic risk has been a necessary and built-in attribute of the system. The neglect 
of the systemic part in the ‘normal mode of operation,’ of course, implies that exter- 
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nal effects are not taken properly into account and that in tendency, market partici- 
pants will ignore the influence of their own behavior on the stability of the system. 
The interesting aspect is more that this was a known and accepted element of oper- 
ations. Note that the blame should not only fall on market participants, but also 
on the deliberate ignoring of the systemic risk factors or the failure to at least point 
them out to the public amounts to a sort of academic ‘moral hazard.’ 

There are some additional aspects as well: asset-pricing and risk management 
tools are developed from an individualistic perspective, taking as given ( ceteris 
paribus ) the behavior of all other market participants. However, popular models 
might be used by a large number or even the majority of market participants. Simi- 
larly, a market participant (e.g., the notorious Long-Term Capital Management) 
might become so dominant in certain markets that the ceteris paribus assumption 
becomes unrealistic. The simultaneous pursuit of identical micro strategies leads to 
synchronous behavior and mechanic contagion. This simultaneous application might 
generate an unexpected macro outcome that actually jeopardizes the success of the 
underlying micro strategies. A perfect illustration is the U.S. stock market crash of 
October 1987. Triggered by a small decrease of prices, automated hedging strategies 
produced an avalanche of sell orders that out of the blue led to a fall in U.S. stock 
indices of about 20 percent within one day. With the massive sales to rebalance 
their portfolios (along the lines of Black and Scholes), the relevant actors could not 
realize their attempted incremental adjustments, but rather suffered major losses 
from the ensuing large macro effect. 

A somewhat different aspect is the danger of a control illusion: The mathematical 
rigor and numerical precision of risk management and asset pricing tools has a 
tendency to conceal the weaknesses of models and assumptions to those who have 
not developed them and do not know the potential weakness of the assumptions and 
it is indeed this that Eichengreen emphasizes. Naturally, models are only approxi- 
mations to the real world dynamics and partially built upon quite heroic assump- 
tions (most notoriously: Normality of asset price changes which can be rejected at 
a confidence level of 99.9999 . . .. Anyone who has attended a course in first-year 
statistics can do this within minutes). Of course, considerable progress has been 
made by moving to more refined models with, e.g., ‘fat-tailed’ Levy processes as 
their driving factors. However, while such models Better capture the intrinsic vola- 
tility of markets, their improved performance, taken at face value, might again con- 
tribute to enhancing the control illusion of the naive user. 

The increased sophistication of extant models does, however, not overcome the 
robustness problem and should not absolve the modelers from explaining their limi- 
tations to the users in the financial industry. As in nuclear physics, the tools pro- 
vided by financial engineering can be put to very different uses so that what is de- 
signed as an instrument to hedge risk can become a weapon of ‘financial mass de- 
struction’ (in the words of Warren Buffet) if used for increased leverage. In fact, it 
appears that derivative positions have been built up often in speculative ways to 
profit from high returns as long as the downside risk does not materialize. Re- 
searchers who develop such models can claim they are neutral academics — devel- 
oping tools that people are free to use or not. We do not find that view credible. 
Researchers have an ethical responsibility to point out to the public when the tool 
that they developed is misused. It is the responsibility of the researcher to make 
clear from the outset the limitations and underlying assumptions of his models and 
warn of the dangers of their mechanic application. 

What follows from our diagnosis? Market participants and regulators have to be- 
come more sensitive towards the potential weaknesses of risk management models. 
Since we do not know the ‘true’ model, robustness should be a key concern. Model 
uncertainty should be taken into account by applying more than a single model. For 
example, one could rely on probabilistic projections that cover a whole range of spe- 
cific models (cf., Follmer, 2008). The theory of robust control provides a toolbox of 
techniques that could be applied for this purpose, and it is an approach that should 
be considered. 

3. Unrealistic Model Assumptions and Unrealistic Outcomes 

Many economic models are built upon the twin assumptions of ‘rational expecta- 
tions’ and a representative agent. “Rational expectations” instructs an economist to 
specify individuals’ expectations to be fully consistent with the structure of his own 
model. This concept can be thought of as merely a way to close a model. A behav- 
ioral interpretation of rational expectations would imply that individuals and the 
economist have a complete understanding of the economic mechanisms governing 
the world. In this sense, rational expectations models do not attempt to formalize 
individuals’ actual expectations: specifications are not based on empirical observa- 
tion of the expectations formation process of human actors. Thus, even when applied 
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economics research or psychology provide insights about how individuals actually 
form expectations, they cannot be used within RE models. Leaving no place for im- 
perfect knowledge and adaptive adjustments, rational expectations models are typi- 
cally found to have dynamics that are not smooth enough to fit economic data well. 5 

Technically, rational expectations models are often framed as dynamic program- 
ming problems in macroeconomics. But, dynamic programming models have serious 
limitations. Specifically, to make them analytically tractable, not more than one dy- 
namically maximizing agent can be considered, and consistent expectations have to 
be imposed. Therefore, dynamic programming models are hardly imaginable without 
the assumptions of a representative agent and rational expectations. This has gen- 
erated a vicious cycle by which the technical tools developed on the base of the cho- 
sen assumptions prevent economists from moving beyond these restricted settings 
and exploring more realistic scenarios. Note that such settings also presume that 
there is a single model of the economy, which is odd given that even economists are 
divided in their views about the correct model of the economy. While other currents 
of research do exist, economic policy advice, particularly in financial economics, has 
far too often been based (consciously or not) on a set of axioms and hypotheses de- 
rived ultimately from a highly limited dynamic control model, using the Robinson 
approach with ‘rational’ expectations. 

The major problem is that despite its many refinements, this is not at all an ap- 
proach based on, and confirmed by, empirical research. 6 In fact, it stands in stark 
contrast to a broad set of regularities in human behavior discovered both in psy- 
chology and what is called behavioral and experimental economics. The corner 
stones of many models in finance and macroeconomics are rather maintained despite 
all the contradictory evidence discovered in empirical research. Much of this lit- 
erature shows that human subjects act in a way that bears no resemblance to the 
rational expectations paradigm and also have problems discovering ‘rational expec- 
tations equilibria’ in repeated experimental settings. Rather, agents display various 
forms of ‘bounded rationality’ using heuristic decision rules and displaying inertia 
in their reaction to new information. They have also been shown in financial mar- 
kets to be strongly influenced by emotional and hormonal reactions (see Lo et al., 
2005, and Coates and Herbert, 2008). Economic modeling has to take such findings 
seriously. 

What we are arguing is that as a modeling requirement, internal consistency 
must be complemented with external consistency: Economic modeling has to be com- 
patible with insights from other branches of science on human behavior. It is highly 
problematic to insist on a specific view of humans in economic settings that is ir- 
reconcilable with evidence. 

The ‘representative agent’ aspect of many current models in macroeconomics (in- 
cluding macro finance) means that modelers subscribe to the most extreme form of 
conceptual reductionism (Lux and Westerhoff, 2009): by assumption, all concepts ap- 
plicable to the macro sphere (i.e., the economy or its financial system) are fully re- 
duced to concepts and knowledge for the lower-level domain of the individual agent. 
It is worth emphasizing that this is quite different from the standard reductionist 
concept that has become widely accepted in natural sciences. The more standard no- 
tion of reductionism amounts to an approach to understanding the nature of com- 
plex phenomena by reducing them to the interactions of their parts, allowing for 
new, emergent phenomena at the higher hierarchical level (the concept of ‘more is 
different,’ cf. Anderson, 1972). 

Quite to the contrary, the representative agent approach in economics has simply 
set the macro sphere equal to the micro sphere in all respects. One could, indeed, 
say that this concept negates the existence of a macro sphere and the necessity of 
investigating macroeconomic phenomena in that it views the entire economy as an 


5 For a critique of rational expectations models on epistemological grounds, see Frydman and 
Goldberg (2007, 2008) and references therein. 

6 The historical emergence of the representative agent paradigm is a mystery. Ironically, it 
appeared over the 70s after a period of intense discussions on the problem of aggregation in 
economics (that basically yielded negative results such as the impossibility to demonstrated 
‘nice’ properties of aggregate demand or supply functions without imposing extreme assumptions 
on individual behavior). The representative agent appeared without methodological discussion. 
In the words of Deirdre McCloskey: “It became a rule in the conversation of some economists 
because Tom and Bob said so.” (personal communication). Today, this convention has become 
so strong that many young economists wouldn’t know of an alternative way to approach macro- 
economic issues. 
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organism governed by a universal will. 7 Any notion of “systemic risk” or “coordina- 
tion failure” is necessarily absent from, and alien to, such a methodology. 

For natural scientists, the distinction between micro-level phenomena and those 
originating on a macro, system-wide scale from the interaction of microscopic units 
is well-known. In a dispersed system, the current crisis would be seen as an invol- 
untary emergent phenomenon of the microeconomic activity. The conceptual reduc- 
tionist paradigm, however, blocks from the outset any understanding of the inter- 
play between the micro and macro levels. The differences between the overall sys- 
tem and its parts remain simply incomprehensible from the viewpoint of this ap- 
proach. 

In order to develop models that allow us to deduce macro events from micro- 
economic regularities, economists have to rethink the concept of micro foundations 
of macroeconomic models. Since economic activity is of an essentially interactive na- 
ture, economists’ micro foundations should allow for the interactions of economic 
agents. Since interaction depends on differences in information, motives, knowledge 
and capabilities, this implies heterogeneity of agents. For instance, only a suffi- 
ciently rich structure of connections between firms, households and a dispersed 
banking sector will allow us to get a grasp on “systemic risk,” domino effects in the 
financial sector, and their repercussions on consumption and investment. The domi- 
nance of the extreme form of conceptual reductionism of the representative agent 
has prevented economists from even attempting to model such all important phe- 
nomena. It is the flawed methodology that is the ultimate reason for the lack of ap- 
plicability of the standard macro framework to current events. 

Since most of what is relevant and interesting in economic life has to do with the 
interaction and coordination of ensembles of heterogeneous economic actors, the 
methodological preference for single actor models has extremely handicapped macro- 
economic analysis and prevented it from approaching vital topics. For example, the 
recent surge of research in network theory has received relatively scarce attention 
in economics. Given the established curriculum of economic programs, an economist 
would find it much more tractable to study adultery as a dynamic optimization 
problem of a representative husband, and derive the optimal time path of marital 
infidelity (and publish his exercise) rather than investigating financial flows in the 
banking sector within a network theory framework. This is more than unfortunate 
in view of the network aspects of interbank linkages that have become apparent 
during the current crisis. 

In our view, a change of focus is necessary that takes seriously the regularities 
in expectation formation revealed by behavioral research and, in fact, gives back an 
independent role to expectations in economic models. It would also be fallacious to 
only replace the current paradigm by a representative ‘non-rational’ actor (as it is 
sometimes done in recent literature). Rather, an appropriate micro foundation is 
needed that considers interaction at a certain level of complexity and extracts macro 
regularities (where they exist) from microeconomic models with dispersed activity. 

Once one acknowledges the importance of empirically based behavioral micro 
foundations and the heterogeneity of actors, a rich spectrum of new models becomes 
available. The dynamic co-evolution of expectations and economic activity would 
allow one to study out-of-equilibrium dynamics and adaptive adjustments. Such dy- 
namics could reveal the possibility of multiplicity and evolution of equilibria (e.g., 
with high or low employment) depending on agents’ expectations or even on the 
propagation of positive or negative ‘moods’ among the population. This would cap- 
ture the psychological component of the business cycle which — though prominent in 
many policy-oriented discussions — is never taken into consideration in contemporary 
macroeconomic models. 

It is worth noting that understanding the formation of such low-level equilibria 
might be much more valuable in coping with major ‘efficiency losses’ by mass unem- 
ployment than the pursuit of small ‘inefficiencies’ due to societal decisions on norms 
such as shop opening times. Models with interacting heterogeneous agents would 
also open the door to the incorporation of results from other fields: network theory 
has been mentioned as an obvious example (for models of networks in finance see 
Allen and Babus, 2008). ‘Self-organized criticality’ theory is another area that seems 
to have some appeal for explaining boom-and-bust cycles (cf. Scheinkman and 
Woodford, 1992). Incorporating heterogeneous agents with imperfect knowledge 
would also provide a better framework for the analysis of the use and dissemination 
of information through market operations and more direct links of communication. 
If one accepts that the dispersed economic activity of many economic agents could 
be described by statistical laws, one might even take stock of methods from statis- 


7 The conceptual reductionist approach of the representative agent is also remarkably different 
from the narrative of the ‘invisible hand’ which has more the flavor of ‘more is different’. 



137 


tical physics to model dynamic economic systems (cf. Aoki and Yoshikawa, 2007; 
Lux, 2009, for examples). 

4. Robustness and Data-Driven Empirical Research 

Currently popular models (in particular: dynamic general equilibrium models) do 
not only have weak micro foundations, their empirical performance is far from satis- 
factory (Juselius and Franchi, 2007). Indeed, the relevant strand of empirical eco- 
nomics has more and more avoided testing their models and has instead turned to 
calibration without explicit consideration of goodness-of-fit. 8 This calibration is done 
using “deep economic parameters” such as parameters of utility functions derived 
from microeconomic studies. However, at the risk of being repetitive, it should be 
emphasized that micro parameters cannot be used directly in the parameterization 
of a macroeconomic model. The aggregation literature is full of examples that point 
out the possible “fallacies of composition.” The “deep parameters” only seem sensible 
if one considers the economy as a universal organism without interactions. If inter- 
actions are important (as it seems to us they are), the restriction of the parameter 
space imposed by using micro parameters is inappropriate. 

Another concern is nonstationarity and structural shifts in the underlying data. 
Macro models, unlike many financial models, are often calibrated over long time ho- 
rizons which include major changes in the regulatory framework of the countries in- 
vestigated. Cases in question are the movements between different exchange rate 
regimes and the deregulation of financial markets over the 70s and 80s. In sum- 
mary, it seems to us that much of contemporary empirical work in macroeconomics 
and finance is driven by the pre-analytic belief in the validity of a certain model. 
Rather than (mis)using statistics as a means to illustrate these beliefs, the goal 
should be to put theoretical models to scientific test (as the naive believer in positive 
science would expect). 

The current approach of using pre-selected models is problematic and we rec- 
ommend a more data-driven methodology. Instead of starting out with an ad-hoc 
specification and questionable ceteris paribus assumptions, the key features of the 
data should be explored via data-analytical tools and specification tests. David 
Hendry provides a well-established empirical methodology for such exploratory data 
analysis (Hendry, 1995, 2009) as well as a general theory for model selection 
(Hendry and Krolzig, 2005); clustering techniques such as projection pursuit (e.g., 
Friedman, 1987) might provide alternatives for the identification of key relation- 
ships and the reduction of complexity on the way from empirical measurement to 
theoretical models. Co-integrated VAR models could provide an avenue towards 
identification of robust structures within a set of data (Juselius, 2006), for example, 
the forces that move equilibria (pushing forces, which give rise to stochastic trends) 
and forces that correct deviations from equilibrium ( pulling forces, which give rise 
to long-run relations). Interpreted in this way, the ‘general-to-specific’ empirical ap- 
proach has a good chance of nesting a multi-variate, path-dependent data-gener- 
ating process and relevant dynamic macroeconomic theories. Unlike approaches in 
which data are silenced by prior restrictions, the Co-integrated VAR model gives the 
data a rich context in which to speak freely (Hoover et al., 2008). 

A chain of specification tests and estimated statistical models for simultaneous 
systems would provide a benchmark for the subsequent development of tests of mod- 
els based on economic behavior: significant and robust relations within a simulta- 
neous system would provide empirical regularities that one would attempt to ex- 
plain, while the quality of fit of the statistical benchmark would offer a confidence 
band for more ambitious models. Models that do not reproduce (even) approximately 
the quality of the fit of statistical models would have to be rejected (the majority 
of currently popular macroeconomic and macro finance models would not pass this 
test). Again, we see here an aspect of ethical responsibility of researchers: Economic 
policy models should be theoretically and empirically sound. Economists should 
avoid giving policy recommendations on the base of models with a weak empirical 
grounding and should, to the extent possible, make clear to the public how strong 
the support of the data is for their models and the conclusions drawn from them. 

5. A Research Agenda to Cope with Financial Fragility 

The notion of financial fragility implies that a given system might be more or less 
susceptible to produce crises. It seems clear that financial innovations have made 


8 It is pretty obvious how the currently popular class of dynamic general equilibrium models 
would have to ‘cope’ with the current financial crisis. It will be covered either by a dummy or 
it will have to be interpreted as a very large negative stochastic shock to the economy, i.e., as 
an event equivalent to a large asteroid strike. 
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the system more fragile. Apparently, the existing linkages within the worldwide, 
highly connected financial markets have generated the spill-overs from the U.S. sub- 
prime problem to other layers of the financial system. Many financial innovations 
had the effect of creating links between formerly unconnected players. All in all, the 
degree of connectivity of the system has probably increased enormously over the last 
decades. As is well known from network theory in natural sciences, a more highly 
connected system might be more efficient in coping with certain tasks (maybe dis- 
tributing risk components), but will often also be more vulnerable to shocks and — 
systemic failure! The systematic analysis of network vulnerability has been under- 
taken in the computer science and operations research literature (see e.g., Criado 
et al., 2005). Such aspects have, however, been largely absent from discussions in 
financial economics. The introduction of new derivatives was rather seen through 
the lens of general equilibrium models: more contingent claims help to achieve high- 
er efficiency. Unfortunately, the claimed efficiency gains through derivatives are 
merely a theoretical implication of a highly stylized model and, therefore, have to 
count as a hypothesis. Since there is hardly any supporting empirical evidence (or 
even analysis of this question), the claimed real-world efficiency gains from deriva- 
tives are not justified by true science. While the economic argument in favor of ever 
new derivatives is more one of persuasion rather than evidence, important negative 
effects have been neglected. The idea that the system was made less risky with the 
development of more derivatives led to financial actors taking positions with ex- 
treme degrees of leverage and the danger of this has not been emphasized enough. 

As we have mentioned, one neglected area is the degree of connectivity and its 
interplay with the stability of the system (see Boesch et al., 2006). We believe that 
it will be necessary for supervisory authorities to develop a perspective on the net- 
work aspects of the financial system, collect appropriate data, define measures of 
connectivity and perform macro stress testing at the system level. In this way, new 
measures of financial fragility would be obtained. This would also require a new 
area of accompanying academic research that looks at agent-based models of the fi- 
nancial system, performs scenario analyses and develops aggregate risk measures. 
Network theory and the theory of self-organized criticality of highly connected sys- 
tems would be appropriate starting points. 

The danger of systemic risk means that regulation has to be extended from indi- 
vidualistic (regulation of single institutions which of course, is still crucial) to sys- 
tem wide regulation. In the sort of system which is prone to systemic crisis, regula- 
tion also has to have a systemic perspective. Academic researchers and supervisory 
authorities thus have to look into connections within the financial sector and to in- 
vestigate the repercussions of problems within one institute on other parts of the 
system (even across national borders). Certainly, before deciding about the bail-out 
of a large bank, this implies an understanding of the network. One should know 
whether its bankruptcy would lead to widespread domino effects or whether con- 
tagion would be limited. It seems to us that what regulators provide currently is 
far from a reliable assessment of such after effects. 

Such analysis has to be supported by more traditional approaches: Leverage of fi- 
nancial institutions rose to unprecedented levels prior to the crisis, partly by evad- 
ing Basle II regulations through special investment vehicles (SIVs). The hedge fund 
market is still entirely unregulated. The interplay between leverage, connectivity 
and system risk needs to be investigated at the aggregate level. It is highly likely, 
that extreme leverage levels of interconnected institutions will be found to impose 
unacceptable social risk on the public. Prudent capital requirements would be nec- 
essary and would require a solid scientific investigation of the above aspects rather 
than a pre-analytic laissez-faire attitude. 

We also have to re-investigate the informational role of financial prices and finan- 
cial contracts. While trading in stock markets is usually interpreted as at least in 
part transmitting information, this information transmission seems to have broken 
down in the case of structured financial products. It seems that securitization has 
rather led to a loss of information by anonymous intermediation (often multiple) be- 
tween borrowers and lenders. In this way, the informational component has been 
outsourced to rating agencies and typically, the buyer of CDO tranches would not 
have spent any effort himself on information acquisition concerning his far away 
counterparts. However, this centralized information processing instead of the dis- 
persed one in traditional credit relationships might lead to a severe loss of informa- 
tion. As it turned out, standard loan default models failed dramatically in recent 
years (Rajan et al., 2008). It should also be noted that the price system itself can 
exacerbate the difficulties in the financial market (see Hellwig, 2008). One of the 
reasons for the sharp fall in the asset valuations of major banks was not only the 
loss on the assets on which their derivatives were based, but also the general reac- 
tion of the markets to these assets. As markets became aware of the risk involved, 
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all such assets were written down and it was in this way that a small sector of the 
market “contaminated” the rest. Large parts of the asset holdings of major banks 
abruptly lost much of their value. Thus the price system itself can be destabilizing 
as expectations change. 

On the macroeconomic level, it would be desirable to develop early warning 
schemes that indicate the formation of bubbles. Combinations of indicators with 
time series techniques could be helpful in detecting deviations of financial or other 
prices from their long-run averages. Indication of structural change (particularly to- 
wards non-stationary trajectories) would be a signature of changes of the behavior 
of market participants of a bubble-type nature. 

6. Conclusions 

The current crisis might be characterized as an example of the final stage of a 
well-known boom-and-bust pattern that has been repeated so many times in the 
course of economic history. There are, nevertheless, some aspects that make this cri- 
sis different from its predecessors: First, the preceding boom had its origin — at least 
to a large part — in the development of new financial products that opened up new 
investment possibilities (while most previous crises were the consequence of over- 
investment in new physical investment possibilities). Second, the global dimension 
of the current crisis is due to the increased connectivity of our already highly inter- 
connected financial system. Both aspects have been largely ignored by academic eco- 
nomics. Research on the origin of instabilities, over-investment and subsequent 
slumps has been considered as an exotic side track from the academic research 
agenda (and the curriculum of most economics programs).This, of course, was be- 
cause it was incompatible with the premise of the rational representative agent. 
This paradigm also made economics blind with respect to the role of interactions 
and connections between actors (such as the changes in the network structure of 
the financial industry brought about by deregulation and introduction of new struc- 
tured products). Indeed, much of the work on contagion and herding behavior (see 
Banerjee, 1992, and Chamley, 2002) which is closely connected to the network struc- 
ture of the economy has not been incorporated into macroeconomic analysis. 

We believe that economics has been trapped in a sub-optimal equilibrium in 
which much of its research efforts are not directed towards the most prevalent 
needs of society. Paradoxically self-reinforcing feedback effects within the profession 
may have led to the dominance of a paradigm that has no solid methodological basis 
and whose empirical performance is, to say the least, modest. Defining away the 
most prevalent economic problems of modern economies and failing to communicate 
the limitations and assumptions of its popular models, the economics profession 
bears some responsibility for the current crisis. It has failed in its duty to society 
to provide as much insight as possible into the workings of the economy and in pro- 
viding warnings about the tools it created. It has also been reluctant to emphasize 
the limitations of its analysis. We believe that the failure to even envisage the cur- 
rent problems of the worldwide financial system and the inability of standard macro 
and finance models to provide any insight into ongoing events make a strong case 
for a major reorientation in these areas and a reconsideration of their basic prem- 
ises. 
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Discussion 

Chairman Miller. I want to thank all of you. 

Appropriate Uses of Financial Models 

Let me begin this panel with a question of the earlier panel. 
Some of those responsible, involved in developing economic mod- 
eling now say that the fundamental problem was that the model 
was wrong, there is more data. I don’t think anyone thinks that 
models should be prohibited or people should be prohibited from 
acting on their models for their investment decisions or whatever. 
The extent to which it can be used, it should be used for regulation, 
safety and soundness regulation. Do any of you — what do each of 
you think about whether the models may be improved and will be- 
come reliable, sufficiently reliable to base capital requirements 
on — or do you think that it is so inherently unpredictable that eco- 
nomic forecasts will never become like predicting the movements of 
the planets, that it may be useful for seeing if a financial institu- 
tion is headed towards trouble, but not to say it is got nothing to 
worry about? Any of you. Dr. Berman. 

Dr. Berman. Thank you. I think models definitely have a very 
significant role, not just in finance but in society in general. The 
question is, what aspect of a model are you looking to use. Certain 
models are designed to predict the future. That is always very dif- 
ficult to do. We can predict where the planets are going to go but 
it is very difficult to predict where the stock market is going to go 
today. That is a very small portion of what financial modeling is 
about, predicting the future. Unfortunately, that is what folks 
glean onto when they start thinking about capital requirements. A 
much larger portion of what modeling is about is understanding: if 
something happens to X, what happens to Y? You don’t have to 
predict the future in order to do that, you just need to know the 
relationships between two different things. 

Let’s take an excellent example. The world’s largest insurance 
company entered into massive amounts of credit default swaps that 
ultimately were responsible for their demise. The bet that they 
took might have turned out to be the best bet that they ever could 
have made. We don’t know because those CDSs are probably still 
out there to a certain extent. But they failed to account for the fact 
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that, what would happen if there was a small dip in the value of 
these, and my counterparty asks for collateral? That’s not a matter 
of predicting the future, that’s just understanding this is the way 
that market works. When the value falls, your counterparty asks 
for collateral. They missed that aspect of the model. That had noth- 
ing to do with predicting the future but just in understanding how 
that worked, and that ultimately led to the demise. And you see 
that pervasive through many, many different types of models 
throughout the system. 

Chairman Miller. It does remind me of Yoga Berra’s wisdom 
that predictions are difficult, especially about the future. Mr. 
Rickards? 

Mr. Rickards. Mr. Chairman, I think it is interesting that a 
number of Members and the witnesses today have referred to plan- 
etary motion as an example of models that work, but I will remind 
everyone that from 200 B.C. to 1500 A.D., the model of the uni- 
verse was a geocentric model in which the sun revolved around the 
Earth. And it was obvious because you woke up in the morning and 
the sun came up over here and went down over there, and that was 
not just a religious belief, that was actually a scientific belief, and 
many brilliant mathematicians worked for centuries to write the 
equations. They weren’t automated, of course, but they wrote those 
models, and when people began observing data from improved tele- 
scopes that didn’t conform to that model, they said well, we just 
need to tweak the model a little bit. Instead of these cycles, they 
created epicycles. They were little twirls within the big twirls, and 
they kept going down that path. The model was completely wrong. 
Actually, the model was right, the paradigm was wrong. The un- 
derstanding of how the world worked was wrong. The sun did not 
revolve around the Earth; the Earth revolved around the sun. 

That is my view of today. You can tweak it, you can improve it, 
you can separate the so-called fat tail and zero in on that tail, and 
there is a complex method called GARCH, Generalized 
Autoregressive Condition Heteroskedasticity and variations on 
that. They are all wrong because the paradigm is wrong, because 
the risk is not normally distributed in the first place. So I think 
these are fatally flawed. 

If a hedge fund that is non-systemically important wants to use 
this model, that is fine. They can use voodoo, as far as I am con- 
cerned, but if you are talking about a bank or regulated financial 
institution, they should be prohibited because they don’t work. 

Chairman Miller. Mr. Whalen. 

Mr. Whalen. I agree with them, and also I think Dr. Berman 
made this point. When you are talking about safety and soundness, 
you don’t want to look at a tactical short-term loss possibility, you 
want to look at the worst case, and we see that now with the bank- 
ing industry. By next year, I think we are going to be looking at 
a double- or triple-digit deficit in the insurance fund, and banks are 
going to have to pay that back. No one anticipated that magnitude 
of loss. So what you have is, on the one hand, a marketplace which 
is very short term. They are working on today’s earnings, next 
quarter’s earnings, what have you, and yet over time, since Glass- 
Steagall, we have slowly eroded the limits on risk taking. So the 
models — whether they worked or not is kind of irrelevant. We slow- 
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ly allowed banks to take more and more risk. So I think what we 
have to do first is say, what risk do we want the utility side of this 
industry, the depository, the lending, the cash distribution part of 
banks, to take, and what part do we want to force, for example, 
into the hedge fund community, which is a perfect place for risk 
taking. 

You know, we can’t come up with the answer to your question, 
Mr. Chairman, as to safety and soundness and capital adequacy, 
unless we quantify the risks that the institutions take. I will give 
you an example. Citigroup in 1991 peaked at about three and a 
half percent charge-offs versus total loans. I think they are going 
to get up to about six this time. Now, can you imagine the public 
and market reaction when the large money centers get up to some- 
thing like two, maybe two and a half times their 1990 loss rate? 
But that is how severe of a skew we are seeing. In the Depression, 
we got up to five percent losses on total loans, so we are closing 
in on the 1930s. I don’t think it will be quite that bad, but we will 
see how long we stay there. That is the other question, how long 
will we see those losses? Will it be two quarters or four? This is 
the kind of question you need to answer very precisely but the only 
way you can answer your question about capital and safety and 
soundness is if you first quantify the risk taking, because otherwise 
I don’t think you can get an answer. 

And by the way, we wrote about this last week. I don’t think you 
can ask the markets to give more capital to banks. I think the G- 
20 and Secretary Geithner are wrong. You have to reduce the risk 
taking, because I don’t think the markets would let J.P. Morgan 
have 20 percent capital assets because the returns will be too low. 
It would be low single digits at best and on a risk-adjusted basis 
I think they would be negative. This, by the way, is the context you 
ought to bear in mind. Most of the large banks on a risk-adjusted 
basis really aren’t that profitable. It is only the super-normal re- 
turns that they get from OTC derivatives, investment banking, pro- 
prietary trading that helped the whole enterprise look profitable. If 
you look at the retail side, the cash securities trading, it is barely 
profitable, really, and that is why you have seen the changes in the 
industry that you have. 

Chairman Miller. Dr. Colander. 

Dr. Colander. In answer to your question, in social science you 
will never get the amount of exactness that you will get in natural 
sciences, mainly because the agent in social science is not an atom 
which sort of follows a set of rules, you know, it is a human being, 
it is an agent who will try to do everything he can to screw you 
every time you are trying to control him. So the thought that you 
are going to be able to design any model is pretty much impossible. 
That being said, I think models can be used and have to be used. 
We all use models. How can you not sort of picture what is going 
on? The question is, what type of models, and how many different 
models do you have in your mind, and how quickly can you jump 
from one model to another and recognize we have really moved 
there, and that is the issue that I think people are talking about. 
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Proposals for Avoiding Recurrences of Financial 
Problems 

Chairman Miller. Interesting set of answers. Mr. Rickards did 
mention at least three proposals for avoiding a catastrophe like 
what we have had. Do the rest of you have specific proposals as 
well of how we avoid this again? I think the financial industry is 
already treating what happened last September, October — we are 
still in it — as a hiccup, something that was a fluke, will not happen 
again, we don’t have to change conduct very much. I assume all of 
you don’t agree with that, but what is it that we should do? 

Dr. Berman. I think there are two courses of action. I think most 
of the discussion on regulatory capital is trying to solve a symptom 
as opposed to the cure itself. A good portion of the funds come from 
investors who are feeding the big bonuses, let’s say, at large banks, 
so while there is lots of talk about the restriction on bonuses and 
whether we should hold people legally liable for clawbacks, et 
cetera, the fact is that the fuel is there. That fuel causes crisis. The 
fuel is done generally by greed, but mostly uninformed greed. Prob- 
ably the number one thing that regulators can enforce is better 
transparency and better disclosure on finance itself. If more people 
understood what they were actually buying, less people would buy 
these things. Wall Street is a marketing arm as are all commercial 
companies. Their practices came from the desires of people to in- 
vest in those products, invest in those services, and invest in the 
companies themselves. If we don’t like those practices, then we 
should make it clear what those practices are and let investors 
choose whether or not they want to engage in those. That would 
dampen further — well, certainly it would help reverse this crisis a 
bit, and it would certainly dampen the ability for the market to 
even create these very, very large bubbles in the first place. 

Mr. Whalen. One simple thing that I would add to Dr. Berman’s 
comment, and speaking as an investment banker, make the law- 
yers your friend. What you want to do is, instead of allowing banks 
to bring these structured assets and derivatives in an unregistered 
forum, you force them to register with the SEC, and what that does 
is two things. First off, the lawyers of the deals will not allow more 
than a certain degree of complexity, because once that deal is reg- 
istered, it can be purchased by all investors, and so they will force 
simplicity onto their banks. Because otherwise they will get sued, 
and the trial lawyers will enforce this, believe me. Remember, most 
of the toxic waste, the complex structured assets, were all done at 
private placements. You can’t even get a copy of the prospectus. 

The second thing I would tell you is that, you know, in terms of 
overall market structure, we’ve got to decide whether or not, going 
back to my earlier comment, we are going to allow people to con- 
trive of any security for any investor that doesn’t have some ration- 
al basis, some objective basis in terms of valuation, because that 
is really the key problem that we have all faced over the last cou- 
ple years, is valuation. When the investors realized that they 
couldn’t get a bid from the dealer that sold them the CDO and they 
couldn’t value it by going to anybody in the cottage community, 
they just withdrew from the market and we had a liquidity prob- 
lem. If you force these deals to be registered, guess what? Every 
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month when the servicer data for the underlying becomes avail- 
able, they will have to drop an 8K and then that data will be avail- 
able to the community for free. We won’t have to spend hundreds 
of thousands of dollars a year to buy servicer data so that we can 
manually construct models to try and understand how a very com- 
plicated mortgage security, for example, is going to perform. You 
will open up the transparency so that the cottage industry that cur- 
rently supports valuation for simple structures, which are very 
easy to value — credit card deals, auto deals — there is really no 
problem with these and they are coming back, by the way. You are 
starting to see volume come back to that market. It is about disclo- 
sure. I think Dr. Berman says it very well. 

Abuse of the VaR 

Chairman Miller. Dr. Colander? You don’t have to speak on 
every topic if you don’t want to. 

Dr. Berman, everyone agrees that the VaR can be abused, has 
been abused, was certainly used foolishly in lowering capital re- 
quirements for investment banks. Without revealing proprietary in- 
formation, can you give us some of the ways that you have seen 
firms abuse the VaR, or try to abuse the VaR apart from regulatory 
matters? 

Dr. Berman. Sure. I don’t think that VaR in itself was purpose- 
fully or willfully abused. VaR is a model that requires a significant 
number of assumptions. For example, if I buy a product, such as 
an option, then I should assume that if the value of the stock goes 
down, then the value of the option will go down. If I write that op- 
tion, so I sell it, then if the stock goes up, I can lose a lot of money. 
If I don’t have the desire or the technology or the capability or the 
incentive to bother being careful about that, then I will assume 
that, if the stock goes up, I will make or lose a limited amount of 
money. That is a very, very poor assumption, which I think we 
have heard a lot today. If you take many of those poor assumptions 
and you add them up, you wind up getting VaR numbers, and not 
just VaR numbers but numbers of all sorts of different models that 
wind up being all but meaningless because of so many small poor 
assumptions that have added up into something that is just wildly 
incorrect. But folks like to believe their own numbers, especially 
when those numbers allow them to do things that they weren’t able 
to do before. So it wasn’t a willful misconduct as much as a care- 
lessness, given the incentive structures that are out there today. 

Chairman Miller. Anyone else? Mr. Rickards. 

Past Congressional Attempts to Regulate the 
Financial Industry 

Mr. Rickards. Yes, Mr. Chairman, I just want to say that my 
recommendations, if we are going back to something like Glass- 
Steagall, there was more to that than just a walk down Memory 
Lane. I am not saying, gee, the system today has obviously failed, 
let us go back to what we had before. I actually derived these from 
my own research into the power load relationship that I talked 
about earlier, which is that scale — as scale goes up, as you triple 
or quadruple or increase by five or ten times the system, you are 
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increasing risk by a factor of 100, 1,000, 10,000. That is the non- 
linear relationship that Dr. Taleb talked about earlier, and I very 
quickly came to the conclusion — well, if that is the problem, then 
descaling is the answer, and Glass-Steagall is an example of that. 
There is a little bit, I think, of — you know, easy with hindsight, but 
perhaps some arrogance in the 1998-2001 period where I think 
Members looked back at the Congress in the 1930s and said, you 
know, they were Neanderthals, they didn’t understand modern fi- 
nance, they created this system. The Members of Congress in the 
1930s had actually lived through something very similar to what 
we are living through now and this was their solution. They actu- 
ally had firsthand experience. 

Now, did a Member of Congress in 1934 understand fractal 
mathematics? No, it was invented in the 1960s. But they had an 
intuitive feel for the risks and I think their solution — we had a sys- 
tem that worked from 1934 to 1999, for 65 years. When the savings 
& loan (S&L) crisis happened in the early 1990s, it didn’t take 
hedge funds with it. When we had the banking crisis in the mid- 
1980s, it didn’t affect the S&L industry or it didn’t affect invest- 
ment banking. We were compartmented, and that is what saved 
the system. We have torn down all the walls. Commercial banks 
look like hedge funds. Investment banks look like hedge funds. 
Hedge funds originate commercial loans. It is a big business for 
them. So when everyone else is in everyone else’s business, should 
it come as any surprise that if one part fails, it all fails. 

Chairman Miller. Thank you. 

Should a Government Agency Test Financial 
Products for Usefulness? 

Dr. Taleb earlier suggested that there be something like the FDA 
that approves — actually it was not clear to me in the earlier panel 
to what extent they were calling for government conduct or setting 
rules by government that would prohibit things, or just people not 
doing them because they were stupid, but assuming we are talking 
about rules that may be set by government, Dr. Taleb suggested 
that the FDA reviews drugs to see if they do any good, they don’t 
allow — the FDA doesn’t allow patent medicines mixed up in a bath- 
tub to be sold to cure cancer anymore. You could do all that you 
wanted in the 1930s. You can’t do it now. And a great many of the 
financial instruments that led to all this have no readily apparent 
social utility and create enormous risk that is dimly understood by 
even the people who are selling them, certainly the CEOs and the 
boards of directors of their institutions. Should we be reviewing fi- 
nancial instruments for whether they have any useful purpose, and 
can you give examples of instruments that have no apparent pur- 
pose and have done great damage? Mr. Whalen. 

Mr. Whalen. Well, I think the short answer is no. I am not a 
big fan of regulation. I don’t think the government has the com- 
petency to analyze complex securities in the first place. You would 
have to hire the people that do it. I think it is better to let the mar- 
ket discipline this behavior. Large buy-side investors, who I would 
remind you are probably the survivors of this period, they are the 
ones with the money, they tell the sell side what they want and 
they are going to tell the rating agencies what they want to see as 
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well, and if you increase the liability to the issuers by forcing dis- 
closure, by forcing SEC registration, you are going to see simplicity. 
Because otherwise my friends at the trial bar are going to come 
over the hill like the barbarians, and they are going to feast, and 
I think that is the way you do it. You don’t want to get the govern- 
ment into a role where they have to make judgments about securi- 
ties, because, frankly, who would you ask? The folks at the Fed? 
I mean, the Fed is populated by monetary economists who couldn’t 
even work on Wall Street. I mean, I love them dearly, I go fishing 
with a lot of these people but I would not ever let them have any 
operational responsibility because they just don’t have the com- 
petency. 

So I think we have to try and take a minimalist approach that 
is effective, and the way you do that is by making the issuer retain 
a portion of the deal that they bring so that they have to own some 
of the risk. You make them make a market in these securities too. 
They can’t just abandon their clients when they bring some com- 
plex deal and not even make a bid for it. That is a big part of the 
problem. If you make the dealers retain some risk and retain re- 
sponsibility, then I think you will see change. 

Chairman Miller. Dr. Colander. 

Dr. Colander. I wanted to expand a little bit on regulation from 
a different perspective, again, agreeing very much with what Mr. 
Whalen said, that there is a problem with government regulation, 
and we can go back and think about Glass-Steagall. You know, peo- 
ple responded to Glass-Steagall and said here is the problem, you 
know, that we deregulated. The problem was, during that time 
there was enormous technological change. We had to change the 
regulations, and now you have to — regulation isn’t a one-time 
thing. It has got to be continually changed, and here is the prob- 
lem. My students, when we asked how many were going on, you 
know, sort of — Paul Volker came up and spoke and he said, you 
know, big audience, “How many of you are planning to go on and 
work for government?” and I think two people raised their hand. 
Then he said, “How many people are planning to go on to Wall 
Street?” You know, you had all this large number, and this was a 
number of years ago. When my students coming out of Middlebury 
College as seniors can earn $150,000 to $200,000 in the first or sec- 
ond year and somebody coming into government can get, what, as 
a GS-8 or 9, you know, sort of $34,000 or something. You know, 
where are you going to go, how are you going to get the expertise 
to do it? And so what happens is, you know, you have an unfair 
system there, where no matter how much regulation you get, given 
the pay structure, given what’s there, the people who are having 
it designed will be able to snow anybody who is trying to regulate 
it, and that is why very much I think you have to design it, not 
so we have to regulate it, but it is self-regulatory, and that, I think, 
is what you are hearing from people, that you have responsibility. 
If it’s too big to fail, we have to regulate it so therefore let us see 
that is not too big to fail by making it smaller, that we structure 
it by the people who know the institutional structure, so that here 
you figure why you won’t make that deal. But not for government 
to be coming in mainly because government will get beat. 

Chairman Miller. Mr. Rickards. 
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Mr. Rickards. Mr. Chairman, I think the idea that there would 
be a government panel of some kind that would vet and approve 
financial products in the manner that the FDA approves drugs is 
probably not workable, probably beyond the ability of government. 
But for example, credit default swaps: There is actually a use for 
them. They are socially useful when they are used to hedge a posi- 
tion in the underlying bond, but they become a casino ultimately 
underwritten by the taxpayers when they are used with no insur- 
able interest. So it is hard enough understanding what a credit de- 
fault swap is, but to get that distinction just right, when it may 
or may not be useful, would be extremely difficult. But I do believe 
there should be a quarantine in the sense that — let’s have these 
products in hedge funds, in long-run investors or maybe with mild 
leverage. Let us keep them out of FDIC-insured banks and other 
institutions that perform this utility function and are in effect gam- 
bling with taxpayers’ money. 

I also endorse Dr. Colander’s suggestion that, in the National 
Science Foundation, in the peer review process, there is a rule for 
looking at these things, perhaps not in the regulatory sense of ap- 
proving them but in the academic sense of understanding them. 
And I believe what Dr. Colander is referring to is what I call ‘cog- 
nitive diversity.’ Let’s just not have a bunch of economists or, for 
that matter, a bunch of physicists but let us have physicists, econo- 
mists, behavioral scientists, psychologists and others work to- 
gether. I think it is interesting that Dr. Kahneman at Princeton 
won the Nobel Prize in economics a few years ago. He is the world’s 
leading behavioral psychologist. He wouldn’t describe himself as an 
economist, but he made very valuable contributions to economics. 

If you get 16 Ph.D.’s in a room and they all went to one of four 
schools, let us say Chicago, MIT, Harvard and Stanford, and they 
are all fine schools, you will actually improve the decision-making 
if you ask two or three of them to leave and invite in the first cou- 
ple people who walk down the street. You will lower the average 
IQ, but you will improve the overall outcome because those people 
will know something that the Ph.D.’s don’t. So at the National 
Science Foundation level, to encourage that kind of collaboration I 
think is very valuable. 

I have actually — I am involved in a field called econophysics 
which basically is understanding economics using some physics 
tools, and I don’t claim it is the answer to all these things but it 
does make some valuable contributions. But when I speak to — I 
have spoken at Los Alamos and the Applied Physical Laboratory, 
and I get a very warm reception. The physicists are very intrigued 
and they see the applications. When you talk to economists, they 
have no interest. They are like, what do physicists have to tell us. 
And I think more collaboration would be helpful. 

Identifying Firms That Are ‘Too Big to Fail’ 

Chairman Miller. Mr. Rickards, you know from personal experi- 
ence that it is not just depository institutions that are systemically 
significant. How do we identify — I think you and Dr. Colander both 
have spoken about the problem of scale. How do we reduce the size 
of institutions? How do we identify the ones that are systemically 
important, either because of their size or their interconnectedness, 
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as inappropriate for the kind of risk taking . . . that if we assume 
that there are some institutions, most hedge funds, that can be 
born and die without any great consequence to the rest of the plan- 
et, and that if they want to use voodoo, they can. How do we iden- 
tify those that we have different standards for? Mr. Whalen. 

Mr. Whalen. Well, I think there is two simple answers. First off, 
we have to revisit market share limits. You have already seen this 
in process with the FDIC because they have started to levy pre- 
miums against total assets, less capital, instead of domestic depos- 
its. I think that is a very healthy change. But perhaps more impor- 
tant, we have to let institutions fail, because if you convince inves- 
tors that you are going to put a Lehman Brothers or a Washington 
Mutual into bankruptcy, they are going to change their behavior, 
and I think both of those events were inevitable, by the way. I 
think it is ridiculous to argue that Lehman could be saved. They 
were for sale for almost a year. Nobody wanted to buy it. So, you 
know, at the end of the day, if we don’t allow failure, and we don’t 
inoculate our population against risk by letting them feel some 
pain from time to time, then we will repeat the mistake. 

Last point, we have got to get the Fed out of bank supervision. 
Monetary economists like big banks. They love them. I worked in 
the applications area of the Fed in New York. I can’t recall a merg- 
er, a large bank merger that they have ever said no to. I worked 
on the “Manny Hanny” (Manufacturers Hanover Trust) trans- 
action, I worked on the Chemical Bank merger, following that with 
Chase. In each case, you could make a very strong case that those 
were bad mergers. They destroyed value. And then look at Bank 
of America. They had to buy Countrywide because they were the 
chief lender to Countrywide’s conduit. They had no choice. It was 
kind of like J.P. Morgan buying Bear Stearns. There really was no 
choice. But then we have the Fed slam Merrill Lynch into Bank of 
America to save a primary dealer. These are the monetary econo- 
mists saying oh, dear, we want to have a few big primary dealers 
we can manage and deal with, it is easier for us. Well, I think that 
is a really skewed perspective. I would like to see another agency 
responsible for approving mergers of financial institutions that ac- 
tually looks at it on an objective basis and says, is this a good idea, 
because we have got a couple mergers, Wells, Wachovia and Bank 
of America with Merrill Lynch that I am not sure are going to 
work. I think both of those institutions may have to be restruc- 
tured and downsized significantly in the next couple of years. 

Chairman Miller. Mr. Rickards. 

Monitoring and Analyzing Hedge Fund Activity and 

Risk 

Mr. Rickards. Mr. Chairman, on the issue of what is a system- 
ically important hedge fund, at the end of the day there will be 
some element of subjectivity in it — whether a $10 billion hedge 
fund is systemically important, but if you have $9.8 billion, you are 
not. It will be a little bit arbitrary and it can’t be based solely on 
size. It has to be based on the complexity. But the first step is 
transparency. You will never be able to make any informed deci- 
sions like that without good information, and every hedge fund 
manager — I have worked in hedge funds banks and investment 
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banks — they will resist that for various reasons but I don’t under- 
stand why the United States Government couldn’t create a facility 
that would keep that information on a secure basis. We keep mili- 
tary secrets, we keep intelligence secrets, we keep other informa- 
tion confidential. You could have a firm like, you know, IBM Global 
Services that would come in, build a facility. It could be secure, get 
clear people running it, and then just say to all hedge funds, look, 
you have to give us all of your information, all of your positions, 
all of your factors in a standardized format, in an automated for- 
mat once a week, we will keep it in a totally secure environment, 
it will not leak out, but we are going to take that and load it into, 
you know, covariance metrics. We are also going to do that for your 
firm, and we are going to have an idea at that point when you are 
taking systemic risks, and at that point there ought to be an ability 
to intervene. And I agree with Mr. Whalen, it should not be the 
Federal Reserve. They do a lousy job with their primary task of 
preventing inflation, and I don’t know why they have been given 
all these other jobs. But there certainly would be expertise in the 
government to do that much, and then to intervene when nec- 
essary. 

Dr. Berman. Adding that, taking all that data, bringing it to- 
gether — 

Chairman Miller. Dr. Berman. 

Dr. Berman. — and putting it into a large covariance matrix sort 
of sounds like VaR. I mean, that is — so you come back to those 
same questions all the time when you say how do we make pre- 
dictions? This may sound like I am answering the question with 
the same exact question, but the best way to protect against this 
is to ask the question to the bank: what would happen if you 
failed? And then determine what the outcome to society or to the 
economy would be. It is not based on the size of the bank, it is 
based on, look at what would happen, not the probability but if the 
bank fails, if a hedge fund fails, what actually will wind up hap- 
pening, what are the knock-on effects. That requires an enormous 
amount of transparency but you don’t need to necessarily make the 
predictions about that, you just need to follow that thread through. 

Chairman Miller. Dr. Colander. 

Dr. Colander. One of the principles, you know, sort of within ec- 
onomics, is taxes have to have reasons and everything else. And I 
think one of the things that thinking of the economy as a complex 
system brings up is that bigness is, per se, bad, you know, sort of 
an interconnection is, so we have lost the sense that there can be 
a tax on ‘bigness’ so that people can decide but it is built within 
that. And to start thinking that, here, if you have a complex sys- 
tem, you have got to keep a whole number of different elements, 
and the only way you are going to be able to do that — because 
there is enormous pressure to grow — is to somehow design within 
the system a counterweight to that, and so thinking along those 
lines, I think is something that follows thinking of the economy as 
a complex system. 

Chairman Miller. Mr. Whalen. 

Mr. Whalen. I will come back to something Mr. Broun said 
about the community banks because I think it is very important, 
and you all are going to be hearing about this a lot next year. If 
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you are going to tax institutions based on risk, and I think that is 
sound, you start with the FDIC. The big banks should pay more 
than the little banks, and when we see the size of the hole that 
we have to fill in over the next, I don’t know, 25 years from this 
crisis, I think that is going to become a very compelling argument. 
The community bankers are going to be living up here next year 
when they start seeing the estimates for what they have to give up 
in revenue and income to fill in this hole. Remember, we are still 
paying for the S&L crisis. There is still debt out there that we are 
paying interest on. We are going to be paying for this crisis for 100 
years. That is how big the numbers are. So think of that as a load 
on the economy. That is kind of the cost of modeling run amuck, 
and, you know, I am serious about this. We are going to be paying 
for this, the banking industry, consumers, investors in banks are 
going to be paying for this for many, many decades. 

Chairman Miller. We are — Mr. Rickards. 

Mr. Rickards. Just briefly. The inverse of complexity is scale. 
You can have complexity at the small scale, a medium scale or a 
large scale. Failure at the first two will not destroy you. Failure at 
the third may, and so I am not against complexity. There is going 
to be complexity. But, again, Dr. Taleb’s example, an elephant is 
a very complex organism, but if it dies, the entire ecosystem doesn’t 
crash. And so let us keep these things in boxes and reduce the 
scale as the antidote to complexity. 

Chairman Miller. We are at the end of our time, but I want to 
thank all of you for being here. Under the rules of the Committee, 
the record will remain open for two weeks for additional state- 
ments from Members and for answers to any follow-up questions 
the Committee may have for the witnesses. Again, I appreciate 
your willingness to come and testify, and it will be useful to have 
all of you as resources for the future as well. Thank you very much. 
The witnesses are excused and the hearing is now adjourned. 

[Whereupon, at 1:30 p.m., the Subcommittee was adjourned.] 
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